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Research  Site 


The  job  site  was  chosen  as  the  location  for  this  effort.  It  was 
decided  that  on-site  military  personnel  would  perceive  the  device  as 
fulfilling  job  performance  object1ves--rather  than  learning  objectives 
associated  with  supporting  Instructors  In  presenting  school  topics. 


Evaluation 


A  formative  evaluation  was  part  of  the  research  design.  Its  purpose 
was  to  identify  and  correct  hardware  design  deficiencies  and  to  shape  the 
content  to  directly  support  effective  job  performance,  as  defined  by  site 
supervisors. 

The  FSTT  failed  in  minor  ways  several  times  during  its  first  year  of 
use.  Only  one  of  these  failures  required  more  than  a  day  to  identify. 

This  was  an  intermittent  failure  which  took  about  2  months  to  identify  and 
correct . 

The  personnel  on  the  job  site  were  able  to  use  the  trainer  simulator 
after  a  briefing  on  what  it  was  and  how  to  turn  it  on.  Trainee  use  did 
not  require  additional  supervisor  or  instructor  time.  Personnel  on  site, 
especially  newly  assigned  personnel,  used  the  FSTT  extensively  on  three 
shifts  each  day  for  a  period  of  15  months.  They  reported  learning  the 
troubleshooting  skills  on  the  FSTT  that  they  needed  to  perform  maintenance 
on  operational  equipment. 

In  the  summatlve  test,  newly  assigned  personnel  with  1  week  of  FSTT 
experience  performed  significantly  better  on  the  operational  equipment 
than  did  newly  assigned  personnel  without  FSTT  experience.  The  FSTT  per¬ 
sonnel  correctly  repaired  the  faults  inserted  In  the  operational  equipment. 
The  non-FSTT  personnel  correctly  repaired  less  than  half  the  same  Inserted 
faults.  The  FSTT  personnel  made  the  repairs  In  half  the  time  that  non- 
FSTT  personnel  required.  Most  FSTT  personnel  were  consistently  efficient 
In  doing  the  maintenance  work,  but  none  of  the  non-FSTT  personnel  were 
consistent.  Job  supervisor*  cannot  allow  personnel  who  are  inconsistent 
performers  to  work  without  supervision  because  they  may  damage  themselves 
or  the  equipment.  Thus,  newcomers  serve  only  as  helpers,  with  few 
becoming  productive  members  of  the  work  force  during  their  first  enlist¬ 
ment.  Since  most  members  of  the  work  force  In  the  volunteer  services  are 
in  their  first  term,  the  results  of  the  present  effort  are  especially 
significant  In  demonstrating  a  means  of  Increasing  the  effective  size  of 
the  productive  work  force. 

The  Formative  and  Summatlve  Evaluations  were  straightforward  and 
conclusive.  The  micro/graphic  simulator  system  proved  to  be  generally 
reliable  and  acceptable  to  job  users  and  job  supervisors,  and  It  signifi¬ 
cantly  improved  job  performance. 
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SUMMARY 


Objective 

The  objective  of  the  current  effort  was  to  improve  performance  in 
maintenance  units  through  use  of  a  micro/graphic  simulation  system.  The 
simulation  hardware/software  was  to  be  generic  and  low  in  cost.  The  tech¬ 
nique  for  developing  the  content  that  made  the  generic  device  a  simulation 
of  specific  operational  eqipment  was  to  be  simple  and  perfonmable  by  per¬ 
sonnel  who  were  professional  only  in  their  maintenance  skills  on  the 
operational  equipment.  The  product  of  this  effort  was  a  Flight  Simulator 
Troubleshooting  Trainer  (FSTT).  It  takes  its  name  from  the  specific 
operational  equipment  (FB-111A  Flight  Simulator),  the  content  of  vrfiich  was 
developed  on  the  generic  microprocessor/vi sual  storage/display  device.  The 
cost  of  the  generic  hardware  device  was  low  (i.e.,  $20,000)  in  comparison 
to  other  equipment  of  this  type.  The  technique  for  generating  content  was 
simple  and  could  be  performed  by  amateur  photography,  artwork,  and  record¬ 
keeping,  along  with  expert  skills  in  troubleshooting. 


Background 

This  effort  is  part  of  a  broader  research  and  development  (RAD) 
program  on  maintenance  training  simulators  conducted  by  the  Air  Force 
Human  Resources  Laboratory  (AFHRL)  at  Lowry  AFB,  Colorado.  The  work 
reported  here  developed  and  evaluated  the  only  generic  device  in  this 
AFHRL  program.  It  Is  a  generic  device  in  that  the  simulation  content  is 
Independent  of  the  hardware.  It  uses  Image  storage/retrieval/displ ay 
viewers  rather  than  three-dimensional  mock-ups  or  two-dimensional  flat 
panel  representations  of  the  parent  (real)  system.  Thus,  it  is  the  only 
simulator  studied  under  this  general  program  which  uses  a  low  cost  generic 
microprocessor/video  Image  device,  with  system  specific  graphic  content, 
as  a  micro/graphic  simulator  system. 

During  the  time  the  RAD  was  conducted,  costs  of  microprocessors  and 
image  storage/display  devices  were  so  drastically  reduced  that  other 
devices  based  on  physical  verisimilitude  to  the  parent  system  are  no 
longer  in  direct  competition  with  micro/graphic  simulator  systems  for 
procurement  money.  The  micro/graphic  devices  cost  only  a  few  thousand 
dollars,  whereas  other  two-dimensional  and  three-dimensional  devices  cost 
10  to  100  times  that  amount.  The  principal  questions  about  these  low-cost 
micro/graphic  devices  is  whether  they  can  be  used  effectively  in  improving 
maintenance  performance  and  how  content  can  be  developed  for  them  to  make 
them  effective.  In  this  effort,  the  content  was  developed  using  new 
Situational  Interaction  techniques.  Situational  Interaction  creates  a 
problem-solving  environment  for  the  learner,  rather  than  a  procedural 
solution  for  the  learner  to  follow. 


Recommendations 


Users  and  supervisors  approved  of  the  situatlonally  Interactive  FSTT 
content  and  recommended  that  such  devices  be  placed  at  additional  job  sites 
and  in  the  training  school. 

It  Is  recommended  that  further  studies  of  micro/graphic  simulator 
systems  be  implemented.  In  vrfiich  personnel  trained  with  such  systems  are 
allowed  to  work  on  their  own  at  selected  job  sites  to  determine  the 
effects  on  unit  productivity. 

An  additional  recommendation  is  that  the  situational ly  interactive  FSTT 
content  be  recorded  on  a  more  cost  effective  micro/graphic  hardware  system 
than  was  developed  for  delivering  the  content  in  this  effort;  i.e.,  a 
videodisc  system  to  replace  the  microfiche  system  used.  The  cost  of  the 
microfiche-based  system  is  three  times  the  cost  of  a  videodisc-based  system 
and  is  inherently  less  reliable  than  a  videodisc  system  with  the  same 
functional  capability  (effectiveness). 
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PREFACE 
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Robert  Trexler  was  the  Program  Director  of  the  effort.  Principal 
Investigator  for  the  project  was  Dr.  Edgar  L.  Shriver,  President  of 
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I.  INTRODUCTION 


The  Problem 


First-term  Air  Force  personnel  constitute  most  of  the  work  force 
available  in  maintenance  units.  A  major  portion  of  their  first  enlistment 
has  passed  before  electronic  maintenance  personnel  become  consistently 
competent  to  perform  maintenance  on  the  equipment  without  supervision. 

Some  personnel  never  achieve  sufficient  competence  to  carry  their  fair 
share  of  the  workload.  A  statement  that  20  percent  of  the  maintenance 
personnel  do  80  percent  of  the  work  is  met  with  affirmative  nods  in  main¬ 
tenance  units.  This  situation  is  not  new;  it  has  been  a  problem  to  a 
greater  or  lesser  degree  for  decades. 


Job  competence  is  not  achieved  in  school.  Schools  do  not  have  suffi¬ 
cient  amounts  of  job-specific  equipment  to  produce  job-competent  graduates, 
therefore,  they  concentrate  on  basics  and  fundamentals.  The  job  sites, 
then,  must  develop  the  required  job  competence  so  that  personnel  can  per¬ 
form  useful  work  rather  than  merely  be  "helpers."  It  is  generally  agreed 
by  job  site  personnel  that  those  newly  assigned  from  school  know  almost 
nothing  about  doing  their  job  (Chenzoff,  et  al.,  in  press).  At  the  job 
sites,  however,  equipment  cannot  be  taken  out  of  use  for  personnel  to 
practice  on.  They  must  learn  as  helpers  until  they  can  be  trusted  not  to 
cause  damage. 


If  a  low-cost  trainer/simulator  can  provide  synthetic  job  experience 
and  make  personnel  consistently  job  competent  early  in  their  first  enlist 
ment,  the  effective  size  of  the  work  force  will  be  increased.  Such  a 
device  would  solve  a  problem  of  long  standing.  The  present  effort  is 
concerned  with  exploiting  the  potential  of  low-cost  microcomputers  for  a 
practical  and  effective  solution  to  this  problem. 


Background 


Research  and  development  on  maintenance  of  electronic  systems  has  been 
conducted  for  more  than  30  years.  This  RAD  has  dealt  with  methods  of 
training,  analysis,  performance  aiding,  and  performance  measurement.  The 
focus  has  been  largely  on  troubleshooting,  which  is  a  primary  job  activity 
and  one  that  constitutes  a  major  problem.  During  this  period  there  has 
been  a  shift  from  troubleshooting  as  problem  solving  to  procedural i zed 
troubleshooting.  While  both  approaches  are  in  practice  today,  neither  is 
considered  entirely  satisfactory  because  troubleshooting  skill  takes  a 
long  time  to  develop,  and  many  maintenance  workers  never  develop  enough 
proficiency  to  be  more  than  marginally  productive.  Therefore,  most  of  the 
troubleshooting  work  is  done  today  by  the  small  percentage  of  the  work 
force  who  become  proficient--with  most  personnel  remaining  relatively 
unproductive. 
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A  justification  for  continuing  poor  maintenance  performance  that  has 
been  used  for  30  years  is  that  military  equipment  has  become  more  complex. 
Therefore,  the  conclusions  that  follow  indicate  that  automated  test  equip¬ 
ment  must  be  used  to  procedural ize  troubleshooting;  Fully  Procedural i zed 
Job  Performance  Aids  must  be  used  to  procedural ize  troubleshooting;  part- 
task  trainers  must  be  used;  Technical  Orders  must  be  better  organized; 
job  performance  must  be  measured  to  identify  weaknesses;  new  learning 
technology  is  needed;  and  maintenance  simulators  are  needed.  There  is 
substantial  agreement  that  no  satisfactory  solution  to  the  electronic 
maintenance  training  problem  has  been  found.  The  solutions  that  involve 
procedural i zing  troubleshooting  have  not  been  notably  successful.  A 
method  enabling  maintenance  personnel  to  learn  to  solve  problems  without 
job  experience  remains  to  be  developed. 

The  history  of  simulators--their  physical  fidelity,  the  transfer  they 
mediate,  their  use  as  part-task  trainers,  etc. --has  also  been  long  and 
extensive.  Simulators  achieved  their  proven  potential  in  training  (in 
most  people's  minds)  because  flight  simulators  have  been  used  with 
remarkable  success  for  training  pilots.  Compared  with  actual  aircraft, 
flight  simulators  are  less  expensive,  safer,  and  better  suited  for 
controlling  and  measuring  learning  events.  They  are  still  very  high  in 
cost  in  comparison  to  what  Is  practical  to  spend  to  prepare  maintenance 
personnel  for  their  job.  The  RAD  question  for  the  present  effort  was: 

Can  training  simulators  of  some  type  be  produced  at  a  low  enough  cost  to 
support  effective  maintenance  training? 

Maintenance  simulators  have  been  used  as  part  of  RAD  on  training 
methods,  although  not  a  prominent  part.  Methods  used  in  maintenance 
training  RAD  have  tended  toward  devices  that  cost  less  than  computer-based 
maintenance  simulators;  e.g.,  simulations  of  the  troubleshooting  process 
have  often  been  based  on  paper  media,  from  tab  tests  to  symbolic  substi¬ 
tute  tests  to  cardboard  mockups.  It  is  only  in  the  last  few  years  that 
microcomputers  have  been  inexpensive  enough  to  be  considered  as  having  the 
same  cost  magnitude  as  paper. 

The  historical  antecedents  of  the  maintenance  simulator  developed  in 
this  effort  are  found  in  paper  and  graphics  more  than  in  the  traditional 
flight  simulator,  because  an  electronic  maintenance  trainer  simulator  need 
not  support  development  of  the  high  degree  of  psychomotor  skill  that  a 
pilot  requires.  This  fact  impacts  maintenance  simulator  design.  Piloting 
skills  require  coordinated  psychomotor  skills,  and  these  skills  are 
usually  taught  with  three-dimensional  (3D)  simulators  having  high  physical 
fidelity.  Maintenance  skills,  on  the  other  hand,  do  not  have  much  of  a 
psychophysical  component.  They  require  system  concept  understanding  and 
problem-solving  ability,  skills  that  do  not  require  a  device  with  high 
physical  fidelity  for  practice  of  maintenance  actions. 

The  experienced  electronic  maintenance  person  uses  logic  or  functional 
understanding  of  the  system,  along  with  knowledge  that  is  system  specific, 
to  perform  (non-procedural i zed)  troubleshooting.  The  troubleshooter  needs 
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to  know  how  to  formulate  hypotheses,  make  deductions  from  than,  and  check 
the  deductions  against  facts  obtained  by  measuring  outputs  of  functional 
portions  of  the  equipment;  i.e.,  the  hypothetico-deductive  process.  The 
troubleshooter  also  needs  knowledge  of  location  and  appearance  of  parts 
to  be  able  to  "navigate"  through  the  physical  system  in  order  to  obtain 
the  data  from  the  system's  outputs.  The  navigation  requires  specific 
knowledge  of  the  physical  appearance  of  objects  and  knowledge  of  the  sym¬ 
bolic  representations  of  objects  in  schematic  diagrams  and  other  "maps"  of 
the  physical  system.  The  troubleshooter  needs  knowledge  of  failure  proba¬ 
bilities,  accessibility  of  components,  and  safety  factors. 

Some  RAD  projects  and  concepts  are  worth  noting  as  historical  antece¬ 
dents  for  the  way  in  which  the  present  effort  was  approached. 

The  first  known  task  analysis  was  made  on  a  mimeograph  machine  in  1952 
(Shriver,  1978).  Operation  of  the  machine  is  a  procedure  accomplished 
through  a  series  of  steps.  Many  tasks  in  the  world  of  work  are  of  this 
type.  Problem  solving  is  not  similarly  structured.  Shriver,  Fink,  and 
Trexler  (1959)  developed  the  "Functional  Analysis"  technique  to  deal  with 
the  problem-solving  job  of  electronic  troubleshooting.  They  originally 
called  it  "Structured  Cue-Response  Analysis."  When  Shriver  and  Hart  (1975) 
included  this  form  of  analysis  in  the  "Front  End  Analysis"  specification 
for  the  U.S.  Army,  they  modified  it  slightly  and  renamed  it  "Functional 
Analysis." 

The  distinction  between  Task  Analysis  and  Functional  Analysis  is  a 
useful  background  concept.  Task  Analysis  identifies  a  sequence  of  cues 
and  responses  or  step-by-step  procedures  for  performing  a  task.  A 
troubleshooting  task  requires  the  identification  and  replacement  of  a  bad 
component.  Until  the  bad  component  is  identified,  the  task  Is  a  problem 
to  be  solved.  Functional  Analysis  categorizes  the  system  into  levels  of 
assemblies,  components,  parts,  and  establishes  their  input-output  depen¬ 
dencies  to  facilitate  conceptualization  of  the  problem.  It  provides  a 
codification  of  the  cues  and  responses,  but  it  does  not  sequence  them. 

When  formatted  as  block  diagrams,  the  results  of  the  Functional  Analysis 
provide  a  map.  Like  a  road  map,  the  diagrams  do  not  give  the  instructions 
for  how  to  get  from  one  point  to  another.  In  troubleshooting,  the  user 
must  decide  where  to  go  and  what  route  to  use  to  get  there.  The  most 
efficient  route  will  depend  on  conditions,  such  as  ease  of  access,  ease  of 
obtaining  measurements,  and  probability. 

In  the  1959  research  by  Shriver,  Fink,  and  Trexler,  several  simulators 
were  developed  and  used  in  a  series  of  studies  over  a  6-year  period. 

These  training  devices  were  described  and  compared  in  a  report  for  a  Naval 
Training  Equipment  Center  (NTEC)  Conference  in  1975  (Shriver,  1975).  The 
first  devices  in  the  series  had  substantial  physical  fidelity  to  the  parent 
system.  The  later  devices  did  not.  They  consisted  of  diagrams  developed 
through  Functional  Analysis,  placed  on  panels  and  "automated"  with  test 
points  at  all  inputs/outputs.  These  devices  were  the  historical  “grand¬ 
fathers"  of  the  "flat  panel"  devices,  such  as  the  Programmable  Maintenance 
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Simulator  (ECU).  These  devices  and  the  instructional  content  based  on 
Functional  Analysis  were  shown  to  mediate  high  transfer  in  one  study  of 
the  series.  In  this  study,  30  students  received  200  hours  of  training, 
including  hands-on  experience,  on  System  A  (an  Acquisition  Radar)  and  their 
troubleshooting  performance  was  tested  for  40  hours  on  40  malfunction 
problems  (on  System  A).  A  second  group  of  30  students  received  200  hours 
of  training  on  System  B  (a  Tracking  Radar)  plus  24  hours  of  simulator 
training  on  System  A.  They  were  given  the  same  performance  test  on  System 
A.  Though  they  had  never  been  trained  on  System  A  nor  had  they  even  seen 
the  system  before,  they  performed  as  well  as  the  students  who  had  received 
200  hours  of  training  on  System  A. 

The  use  of  graphics  in  simulating  the  job  also  has  a  history  in  per¬ 
formance  testing.  In  the  AFHRL  work  on  "symbolic  substitutes"  (Shriver  S 
Foley,  1974a, b),  performance  tested  on  graphic  representation  of  physical 
entities  (symbolic  substitutes)  was  compared  to  "hands-on"  performance 
tested  on  the  parent  equipment.  As  troubleshooting  tests,  performance  on 
the  symbolic  substitutes  had  a  high  correlation  to  performance  on  the 
system.  However,  the  test  administrators  served  as  random  access  data 
storage  and  retrieval  "devices,"  in  that  they  selected  and  handed  pages  of 
data  to  test  subjects  when  requested.  Although  the  graphic  technique  was 
effective,  the  demands  on  administrators  were  too  great  for  successful 
implementation.  Shriver  and  Foley  ( 1974a ,b)  at  that  time,  called  for 
research  to  develop  a  random  access  machine  to  present  data  graphically 
when  requested  by  the  troubleshooter.  No  low-cost  random  access  machines 
were  available  at  that  time. 

Edwards  (1976)  considered  several  instructional  methods  for  teaching 
problem  solving,  including  the  case  study  or  socratic  method,  the  clinic 
or  Internship,  as  well  as  simulation.  Of  these  three,  she  concluded  that 
simulation  was  the  most  promising  Instructional  method  and,  furthermore, 
that  computer-based  simulation  could  provide  a  flexible  and  individually 
responsive  dimension  tvhll e  at  the  same  time  automatically  scoring  and 
maintaining  records  of  student  performance.  Edwards  felt  that  problem¬ 
solving  skills  are  qualitatively  different  from  other  less  complex  kinds 
of  intellectual  processing.  This  same  notion  was  considered  by  Condon, 
Ames,  Hennessy,  Shriver,  and  Seeman  (1979),  who  discussed  the  need  for 
training  a  wide  variety  of  separate  problems  to  increase  general  problem¬ 
solving  ability. 

McGuire  (1976)  discussed  the  use  of  written  simulations,  both  as  a 
training  method  and  as  a  method  for  diagnostic  testing.  McGuire  contrasted 
simulation  training  with  conventional  programmed  instruction.  She  charac¬ 
terizes  simulation  training  as  follows:  (a)  focus  is  on  the  total  problem, 
(b)  the  student  assumes  full  control  over  the  learning  situation,  (c)  the 
student  acts  as  a  problem  solver,  (d)  feedback  is  informative  about  events 
in  the  problem  situation,  and  (e)  branching  is  clearly  a  consequence  of 
the  student's  own  decisions.  On  the  other  hand,  conventional  programmed 
Instruction  is  seen  as  follows:  (a)  concepts  to  be  learned  are  broken 
into  small  segments,  (b)  focus  is  on  bits  and  pieces  until  each  segment  of 
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understanding  is  error  free,  (c)  carefully  delineated  questions  are  posed, 
(d)  immediate  feedback  is  provided  to  the  student  to  correct  misconcep¬ 
tions  and  to  reinforce  learning  behavior,  and  (e)  the  learner  has  no 
genuine  control  over  some  learning  sequences. 

The  performance  assessment  scheme  developed  by  McGuire  (1976)  for 
evaluating  the  diagnostic  ability  of  medical  students  was  used  as  the 
basis  for  the  extrinsic  feedback  system  in  the  present  study.  Extrinsic 
feedback  represents  information  about  performance  that  would  not  come  from 
performing  on  job  equipment.  In  the  present  case,  it  comes  from  expert 
judgments  about  how  efficient  the  choices  of  actions  were  in  solving  the 
problem.  This  extrinsic  feedback  system  required  experts  to  rate  alter¬ 
native  actions  in  terms  of  their  contribution  to  the  solution  of  the 
problem.  That  is,  some  actions  were  rated  extremely  relevant,  others  less 
so,  and  some  were  unsafe  and  not  permitted.  These  efficiency  ratings  were 
subjective,  and  because  experts  do  not  necessarily  agree,  the  ratings  do 
not  represent  an  absolute  standard  or  even  a  criterion  for  all  aspects  of 
performance.  They  may  indeed  promote  inefficient  learning  by  discouraging 
achievement  of  the  objective,  which  is  learning  to  map  all  paths  to  solu¬ 
tion.  Other  paths  will  be  more  efficient  for  solving  other  problems,  and 
all  paths  will  be  needed  to  learn  the  system. 

Finally,  a  trainer  device  with  physical  characteristics  that  are  very 
similar  to  those  of  the  Flight  Simulator  Troubleshooting  Trainer  (FSTT) 
was  noted  in  the  literature.  Rigney,  Towne,  Moran,  and  Mishler  (1978) 
described  the  device,  originally  referred  to  as  the  Rigney  Trainer.  It 
used  a  microprocessor  and  microfiche  storage  system  to  present  system 
data,  and  a  cathode-ray  tube  (CRT)  display  to  provide  computer-assisted 
instruction  (CAI)  dialogue.  The  learning  strategy  for  this  trainer  was 
instructional  rather  than  experiential  discovery  learning.  However,  in 
terms  of  hardware,  the  FSTT  and  Rigney  Trainer  would  Interchangeably  meet 
the  same  functional  specifications.  At  a  later  date,  this  trainer  was 
designated  as  the  Electronic/Electric  Maintenance  Trainer  (EEMT),  and  a 
videodisc  storage  device  was  substituted  for  the  original  microfiche 
device. 

The  history  of  training  devices  that  mediate  transfer  by  physical  verisi¬ 
militude  of  components  to  a  parent  system  is  not  treated  in  detail  here, 
nor  are  procedural  trainers  or  procedural i zed  jobs.  The  traditional  use 
of  physical  fidelity  and  2D  versus  3D  devices  are  not  relevant  Issues  in 
the  present  effort.  The  process  of  categorizing  tasks  as  cognitive  or 
psychomotor  or  procedural  is  not  a  relevant  issue  because  the  approach  in 
the  present  effort  was  to  avoid  such  categorizations.  The  focus  is  on  the 
cognitive  aspects  of  tasks,  without  denying  that  the  same  tasks  have  or 
could  have  psychophysical  and  specific  knowledge  aspects  as  well. 

The  present  investigation  treats  specific  knowledge  as  incidental 
learning  content  to  be  learned  in  the  context  of  the  larger  objective 
which  is  learning  to  solve  problems.  This  is  contrary  to  abstracting 
specific  knowledges  and  skills  into  categories  or  objectives  for  learning 
outside  the  simulated  job  content. 
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The  research  on  the  instructional  approach  of  keeping  the  whole  job 
content  intact  rather  than  abstracting  it  into  parts  for  training  is  his¬ 
torically  one  of  the  oldest  in  psychology.  It  was  called  the  "part-whole" 
issue  in  the  early  part  of  this  century,  and  such  studies  abounded  in  the 
psychological  literature.  It  split  psychology  into  the  Gestalt  School 
(whole)  and  the  Behaviorist  School  (part).  The  earliest  military  research 
application  was  by  Jones  and  Odom  (1954)  when  the  Human  Resources  Research 
Organization  (HumRRO)  was  first  formed  to  conduct  training  research  for 
the  Army.  The  researchers  found  that  soldiers  consistently  aimed  their 
rifles  high  at  night.  The  training  that  was  developed  on  how  to  aim  at 
night  appeared  to  be  completely  ineffective.  It  was  found  that  because 
soldiers  did  not  believe  they  fired  too  high  at  night,  they  did  not 
understand  the  instruction  on  how  to  fire  lower.  Therefore,  a  simulated 
nighttime  job  experience  was  created  with  silhouette  targets  to  represent 
enemy  soldiers.  Unseen  12-foot  panels  were  placed  behind  the  simulated 
targets  to  collect  the  high  bullet  holes  that  resulted  when  the  soldiers 
fired  at  the  silhouettes.  In  the  simulated  situation,  as  in  the  actual 
job  situation,  soldiers  believed  they  had  hit  the  targets  when  they  aimed 
at  them.  When  they  saw  their  hits  in  the  panels  from  4  to  12  feet  above 
the  heads  of  the  silhouette  targets,  the  instruction  on  how  to  correct 
their  aiming  procedure  became  meaningful.  Each  learned  the  procedures  to 
correct  the  amount  that  they  had  aimed  high.  The  following  night,  the 
soldiers  applied  vrfiat  they  had  learned  and  hit  the  targets  instead  of  the 
panels. 

Shrlver,  Sivy,  and  Rosenqulst  (1955)  used  the  same  approach  a  year 
later  to  train  infantry  squads  in  night  techniques  of  fire.  Night  tech¬ 
niques  required  coordination  of  Individual  fire  to  mass  it  at  the  enemy 
location  using  cues  that  were  different  under  night  conditions.  These 
cues  could  not  be  recognized  from  a  verbal  explanation  without  a  frame  of 
reference.  A  standardized  attack  on  a  squad  position  was  simulated  as  a 
criterion  job  situation.  Squads  of  soldiers  received  experience  in  this 
whole  job  situation  first,  followed  by  successful  instruction  on  proce¬ 
dures  for  using  cues  available  at  night  to  mass  their  fire  to  the  right 
and  left. 

This  instructional  model  was  applied  by  other  HumRRO  researchers  in 
the  1950s.  It  was  applied  to  maintenance  training  by  several  researchers 
Including  Shrlver,  Brown,  Shoemaker,  McKnight,  Gebhard,  and  their  asso¬ 
ciates.  Shoemaker  (I960)  coined  the  name  "functional  context"  to  mean 
that  the  context  of  the  whole  function  is  kept  Intact  as  the  first 
training  experience,  followed  by  detailed  instruction  on  the  various 
aspects  of  the  whole  experience. 

Research  applications  showed  high  student  motivation  and  large 
Increases  in  job  performance  capability.  However,  use  of  this  instruc¬ 
tional  model  more  or  less  ceased  after  showing  great  promise  in  research. 
It  required  large  amounts  of  operational  equipment  for  each  student  to 
experience  whole-job  task  situations,  and  schools  did  not  have  those  quan¬ 
tities  of  equipment.  Now  that  low-cost  microcomputers  and  image  storage 
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devices  can  simulate  functioning  operational  equipment,  the  approach  has 
finally  become  practical.  The  present  effort  is  designed  to  keep  the 
functional  context  of  whole-job  tasks  intact  in  simulations. 

In  the  present  effort  the  instructional  strategy  is  functional  context 
or  learn-by-doing  or  discovery  learning.  The  model  for  the  problem¬ 
solving  situations  is  the  hypothetico-deductive  process,  and  the  technique 
for  creating  the  situations  is  Situational  Interaction  based  on  micropro¬ 
cessor  control  of  random  access  to  images  of  operational  equipment  and 
technical  orders  (TOs). 

This  historical  summary  has  been  an  introduction  to  the  concepts  of 
importance  in  the  present  effort.  In  addition,  several  studies  conducted 
over  the  past  6  years  were  direct  antecedents  to  the  present  work.  They 
were  part  of  an  overall  AFHRL  effort,  which  included  a  literature  survey, 
needs  analysis,  market  survey,  job  analysis  of  flight  simulator  mainte¬ 
nance,  and  development  of  functional  and  procurement  specifications  for 
flight  simulator  maintenance  simulators. 

The  literature  survey  (Condon,  et  al.,  1979;  Fink  A  Shriver,  1978a) 
showed  that  physical  fidelity  was  the  primary  means  used  for  making  simu¬ 
lators  like  the  parent  system.  An  exception  was  the  flat-panel  simulator, 
or  2D  device,  which  jains  its  verisimilitude  by  a  combination  of  graphic 
depictions  on  one  or  more  flat  panels  upon  which  selected  controls  and 
displays  from  the  parent  system  are  mounted.  In  the  literature,  RAD 
issues  focused  on  physical  fidelity  and  transfer.  Some  discussion  of 
psychological  fidelity  was  found  but  without  an  operational  definition  or 
practical  implementation  of  it,  except  in  2D  flat  panel  simulators. 

Little  consideration  of  practical  factors,  such  as  reliability  of  devices 
or  affordability  to  enable  every  student  to  have  one,  was  found  in  the 
1 iterature. 

The  needs  analysis  (Fink  A  Shriver,  1978b)  revealed  that  school  main¬ 
tenance  instructors  perceived  no  need  for  maintenance  training  devices  or 
maintenance  simulators  with  which  they  had  no  personal  experience.  Those 
who  had  no  experience  with  anything  but  real  equipment  saw  no  use  for 
anything  but  real  equipment  or  a  maintenance  simulator  with  extremely  high 
physical  fidelity.  The  respondents  in  the  needs  analysis  were  told  of 
research  results  that  demonstrated  training  effectiveness  of  graphic  simu¬ 
lations  of  real  equipment,  and  the  instructional  concept  of  increasing 
physical  fidelity  of  simulators  with  increasing  skill  levels  was  discussed. 
The  instructors  acknowledged  that  graphics  and  overhead  projectors  or 
simple  plastic  models  were  useful  to  support  their  lectures,  but  could  not 
accept  them  as  objects  on  which  to  practice. 

A  market  survey  revealed  that  there  was  an  abundance  of  off-the-shelf 
components  for  low-cost,  microprocessor-controlled  video  display  devices. 
There  were  Image  storage  devices  based  on  videodisc  (laser),  magnetic 
disc,  and  microfiche  technology.  Videodisc  mastering  processes  were  not 
considered  sufficiently  reliable.  Magnetic  disc  technology  could  not  con¬ 
vert  photographic  images  to  images  stored  on  magnetic  disc.  Microfiche 
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storage  devices  were  available,  simple  to  master  from  photographs,  and  to 
change  or  update. 

The  job  analysis  (Condon,  et  al.,  1979)  revealed  that  troubleshooting 
was  the  principal  main-tenance  job  on  FB-111A  flight  simulators.  The 
troubleshooting  technical  orders  on  this  parent  system  had  not  been 
procedural ized.  Troubleshooting  was  still  essentially  a  problem-solving 
job.  Descriptions  of  about  60  troubleshooting  problem  situations,  repre¬ 
sentative  of  failures  in  flight  simulators  and  obtained  through  records 
and  interviews  with  personnel  Mho  had  performed  the  troubleshooting,  were 
used  to  identify  job  performance  requirements.  Comparing  the  job  perfor¬ 
mance  requirements  to  course  content  in  the  school  revealed  little 
correspondence.  The  course  contained  extensive  treatment  of  basics  and 
fundamentals  useful  only  with  experience.  The  only  practice  available  in 
the  course  was  on  an  obsolete  flight  simulator,  and  very  few  course  hours 
involved  actual  experience  on  the  obsolete  equipment. 

Two  procurement  specifications  were  produced,  one  for  a  relatively 
expensive  simulator  based  on  physical  fidelity,  the  other  based  on  micro¬ 
computer  random  access  to  stored  Images  of  operational  equipment.  It  was 
decided  not  to  procure  the  high  physical  fidelity  device  (in  spite  of  the 
opinions  of  school  Instructors)  because  the  R&D  requirement  was  to  develop 
and  test  the  effectiveness  of  a  conceptual  trainer/simulator.  The  concep¬ 
tual  trainer  was  believed  to  be  particularly  suitable  to  deliver  on-the- 
job  training.  The  lack  of  perceived  need  by  school  instructors  contributed 
to  the  decision  to  Introduce  the  Flight  Simulator  Troubleshooting  Trainer 
(FSTT)  at  the  job  site.  Another  reason  for  field  testing  the  FSTT  at  the 
job  site  was  to  increase  the  probability  of  obtaining  actual  job  perfor¬ 
mance  data  In  the  summatlve  evaluation.  There  was  no  other  place  to 
obtain  performance  data  on  the  parent  system. 

The  specification  called  for  the  use  of  graphics  and  words  exclusively 
to  create  the  verisimilitude  with  the  parent  system.  All  stimuli  repre¬ 
senting  the  parent  system  and  its  technical  documentation  were  to  be 
presented  on  viewing  screens.  All  responses  by  users/students  were  to  he 
input  to  the  computer  through  a  device  such  as  the  hexidecimal  pad. 

This  concept  was  a  distinct  departure  from  the  tradition  of  trainers/ 
simulators  that  gain  fidelity  through  hardware  similarity  or  through 
spatial  orientation  of  objects.  In  terms  of  physical  fidelity,  flat-panel 
and  other  types  of  2D  trainers  are  midway  between  the  older  tradition  and 
the  device  developed  in  the  present  effort.  Flat-panel  devices  have  often 
been  selected  for  maintenance  training  because  of  their  suitability  for 
training  maintenance  skills  and  because  they  are  less  costly  than  3D 
trainers  and  they  can  support  sophisticated  instructional  strategies  and 
performance  assessment  schemes.  They  have  also  been  used  as  procedural 
trainers  (Pleper  S  Benson,  1975).  But  regardless  of  how  they  are  used, 
the  2D  flat-panel  device  Is  expensive  in  comparison  to  devices  which  use 
microprocessors  to  select  graphic  Images  for  display  on  screens. 
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The  availability  of  low-cost  microcomputers  now  makes  conceptual 
trainers  for  electronic  troubleshooters  practical.  In  the  past  few  years 
several  such  trainers  have  been  developed  in  research  for  the  Navy  and 
Army.  In  fact,  any  number  of  variations  in  such  trainer  simulators  can  be 
constructed  by  cabling  together  components  available  from  any  number  of 
vendors.  With  all  such  combinations  of  hardware,  it  is  the  courseware 
used  to  provide  the  content  of  the  training  that  makes  the  hardware  a 
trainer/simulator.  It  is  also  the  courseware  and  learning  strategies  that 
determine  what  kind  of  simulator  it  will  be. 

The  present  trainer/simul ator  is  an  extremely  simple  device  built  of 
high  quality  components.  The  software  or  application  program  is  burned 
into  a  memory  (EPROM)  chip.  It  is  designed  to  require  no  special  computer 
programming/authoring  skills  on  the  part  of  the  person  who  prepares  the 
courseware  troubleshooting  problems.  Lessons  for  entirely  different 
parent  systems  can  be  prepared  for  the  same  hardware/ software  device.  The 
device  itself  is  not  system  specific— only  the  courseware  content  is 
system  specific.  The  trainer/simulator  gains  its  verisimilitude  to  the 
parent  system  from  graphic  content,  not  from  hardware  features. 

The  trainer/simulator  is  based  on  what  has  been  learned  in  previous 
RAD  efforts.  The  RAD  has  been  quite  extensive,  it  has  shown  good  results, 
and  it  leads  precisely  to  the  approach  made  possible  when  the  random 
access  microprocessor  became  sufficiently  low  in  cost  to  make  the  approach 
practical . 
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II.  APPROACH 


Simulation  Approach 

The  FSTT  is  a  low-cost,  stand-alone  interactive  graphic  simulator 
which  permits  simulation  of  the  conceptual  aspects  of  system  trouble¬ 
shooting  and  repair  within  the  format  of  problem-solving  exercises.  The 
physical  medium  is  visual  images  displayed  upon  independently  addressed, 
dual  random  access  micrographic  projectors.  User-simulator  interaction  is 
permitted  by  means  of  a  16-key  keypad  and  microprocessor  control.  Magnetic 
cards  are  used  to  select  problems  and  to  control  the  in-line  display  feed¬ 
back.  A  printer  provides  records  of  student  performance.  Courseware 
consists  of  simulations  of  troubleshooting  problems  represented  by  photo¬ 
graphs  of  actual  equipment  components,  displays,  and  documentation. 

As  of  this  writing,  the  random  access  hardware  developed  to  implement 
the  situational ly  interactive  courseware  is  obsolete.  The  random  access 
capability  is  available  now  from  other  sources,  most  notably  the  videodisk 
microcomputer  combination.  The  FSTT  is  described  in  detail  in  the 
following  pages  for  its  historical  interest  and  to  provide  a  record,  as 
well  as  to  give  the  functional  specifications  for  programming  other  media 
such  as  videodisk/microcomputer  hardware  to  accomplish  the  same  functions. 

In  a  previous  study,  school  instructors  were  asked  about  their  needs 
for  simulators.  They  reported  a  need  for  small  training  devices  to 
illustrate  their  lectures  but  did  not  perceive  a  need  for  a  simulator  such 
as  the  one  developed  in  this  effort. 

The  reason  the  instructors  did  not  perceive  this  need  provides  the  key 
to  understanding  the  need  that  the  present  simulator  is  designed  to  fill. 
The  traditional  process  for  developing  instructional  content  implicitly 
accepts  the  fact  that  the  real  job  situation  cannot  be  made  available  in 
the  school.  Therefore,  techniques  have  been  developed  for  abstracting  job 
requirements  from  the  actual  job  situation  so  they  can  be  presented  by 
instructors.  Traditionally,  the  means  used  for  abstracting  is  to  deduce 
assumptions  about  skill  and  knowledge  requirements  of  the  job.  The 
instructor  then  uses  a  verbal  medium  to  deliver  instructions  on  those 
skills  and  knowledge.  Instructors  can  illustrate  their  lectures  with  the 
aid  of  graphic  media,  and  they  can  use  simple  mockups  and  other  devices  in 
support  of  the  instruction.  Therefore,  when  school  instructors  are  asked 
what  simulators  they  need,  their  interpretation  of  need  must  be  in  terms 
of  devices  to  support  their  lectures. 

The  well  known  instructional  systems  development  (ISD)  methodology 
provides  ?  more  formal  process  of  deriving  task  requirements  from  the  job. 
The  requirements  are  stated  in  terms  of  terminal  and  enabling  objectives 
that  describe  the  job  task  actions.  Alternative  media  to  choose  from  vrfien 
using  ISD  processes  are  substantially  increased  over  those  available  to 
the  classroom  instructor.  One  of  about  a  dozen  media  for  presenting 


11 


Instruction  is  a  simulator.  Within  ISO  the  simulator  is  considered  as  an 
instructional  medium  to  be  used  after  the  analytical  abstraction  of 
requirements  from  the  job  situation. 

The  simulator  built  and  tested  in  the  present  effort  was  not  designed 
as  an  instructional  medium  but  as  a  performance  criterion  situation  in 
which  learning  could  take  place.  The  simulator  is  designed  to  provide 
synthetic  experience  in  repairing  the  prime  equipment.  That  job  involves 
a  broad  spectrum  of  job  knowledge  and  skills.  The  job  requirements 
Include  learning  the  location  of  the  displays  and  controls,  their  normal 
and  abnormal  appearances,  where  to  go  to  find  test  points,  what  components 
are  accessible,  what  activities  take  more  time  than  others  to  perform, 
which  activities  are  dangerous  to  perform,  which  functions  are  dependent 
on  which  other  functions,  how  to  deduce  what  parts  have  an  effect  on  other 
parts'  outputs,  etc.  The  simulator  was  designed  to  have  these  job  require¬ 
ments  Imbedded  such  that  the  learner  discovers  and  masters  them  in  the 
single  context  of  the  simulated  job  situation.  This  is  possible  because 
the  simulation  is  a  detailed  representation  of  the  job  and  requires  no 
separate  instruction.  The  simulation  design  is  based  on  the  assumptions 
of  discovery  learning  or  learn1ng-by-doing--the  way  that  people  learn  by 
experience  on  the  actual  job. 

The  intent  of  the  present  effort  was  to  exploit  the  existing  capabili¬ 
ties  of  low-cost  microcomputers,  rather  than  to  design  new  hardware,  to 
meet  job/training  requirements.  The  capability  of  low-cost  microcomputers 
to  access  job  task  situations  quickly  In  graphic  displays  was  the  principal 
capability  that  made  the  present  simulator  possible.  The  design  objective 
was  to  make  a  simulation  as  much  like  the  job  as  possible  with  available 
graphic  image  projectors  and  microprocessor  random  access  capabilities. 

The  job  features  that  can  be  simulated  with  the  low-cost  hardware  com¬ 
binations  are  sensory  dl sc rml nations  and  cognitive  aspects  as  opposed  to 
psychomotor  aspects.  In  this  study,  it  was  clearly  recognized  that  there 
are  some  psychomotor  aspects  to  the  job  being  simulated.  It  was  also 
recognized  that  there  were  job/training  requirements  for  knowledge  of 
locations  of  operational  equipment  components.  The  simulator  hardware  has 
the  capability  of  providing  views  of  these  components  in  the  context  of 
their  locations. 

The  troubleshooting  problems  provided  by  the  courseware  were  designed 
to  require  users  to  look  at  overall  views  of  the  operational  equipment, 
choose  a  location  to  examine  next,  from  that  view  choose  the  next,  etc. 

The  visible  cues  that  users  would  confront  in  the  real  situation  were  all 
represented  graphically. 

The  simulator  was  designed  with  two  screens  so  users  could  see  simul¬ 
taneously  a  view  of  where  they  had  been  and  a  view  of  what  they  might 
choose  to  Inspect.  The  dual  screen  design  was  partially  derived  from  the 
requirement  for  learning  locations  from  two-dimensional  views  of  opera¬ 
tional  equipment.  Two  screens  also  made  it  possible  to  simulate  using  TO 
documentation  vrtiile  looking  at  the  physical  equipment. 
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Specific  job  characteristics  are  simulated  with  the  courseware 
problems  rather  than  with  hardware  characteristics.  Since  the  courseware 
problems  were  designed  to  keep  as  much  as  possible  of  the  job  situations 
intact  In  their  job  context,  the  learning  situations  were  also  the  cri¬ 
terion  tests  of  performance. 


The  Training  Approach 

The  FSTT  trainer/simulator  system  places  students  in  a  problem 
situation  and  allows  them  to  work  their  way  to  a  solution  of  the  problem 
using  the  strategy  of  discovery  learning.  There  are  many  elements  of 
specific  knowledge  required  to  solve  a  problem  and  even  more  to  solve  it 
efficiently.  The  FSTT's  training  strategy  meets  the  problem-solving 
objective  and  also  provides  practice  in  discovering  and  using  specific 
knowledge. 

The  scientific  hypothetico-deductlve  model  is  the  basis  for  problem 
solving  using  the  FSTT.  In  this  model  the  student  forms  a  hypothesis 
after  being  confronted  with  the  situation  (data,  symptoms,  conditions, 
etc.).  The  student  then  selects  actions  that  produce  situations  from 
which  the  student  will  obtain  collect  data  that  support  or  negate  the 
hypothesis.  Based  on  this  information,  a  more  specific  hypothesis  Is 
formed,  and  the  process  continues  until  a  final  solution  Is  reached.  The 
solution/conclusion  is  tested  by  replacing  the  part  identified  as  faulty 
and  confirming  that  the  replacement  corrected  the  malfunction. 

Learning  problem  solving  by  the  hypothetico-deductlve  process  is 
accomplished  in  the  present  simulator  through  a  technique  called 
Situational  Interaction.  Utilization  of  the  technique  requires  random 
access  hardware  capability.  Images  of  situations  that  provide  information, 
choices  of  actions  to  be  taken,  and  the  means  for  the  student  to  input 
choices  to  the  random  access  computer. 

The  images  that  constitute  the  courseware  designed  for  the  present 
application  enable  students  to  practice  the  hypothetico-deductlve  process. 
The  student  learns  while  practicing,  thus  discovering  how  to  solve  the 
problem  and  also  how  to  become  more  efficient  in  solving  it.  The  course¬ 
ware  provides  some  aids  to  solution  that  are  lists  of  reasonable  actions 
to  take  in  a  situation,  but  no  instructions  are  given  on  what  choices 
should  be  made.  In  solving  each  problem,  the  student  learns  many  speci¬ 
fics.  Most  are  concerned  with  the  parent  system— what  symptoms  look  like, 
where  they  are  found,  where  test  points  are  locate d,  which  test  points 
require  the  least  effort  to  gain  access  to,  etc.  Last  but  not  least,  the 
student  learns  something  about  the  probability  of  failure  for  various 
components  in  the  parent  system.  All  of  this  specific  learning  Is 
accomplished  In  the  context  of  the  general  learning  strategy. 

The  problems  are  taken  from  all  the  different  functional  areas  of  the 
parent  system  so  the  initial  symptoms  in  each  problem  will  be  unique;  e.g., 
pneumatic  leaks,  audio  communications  failures,  visual  display  symptoms. 


control  malfunctions,  etc.  After  the  Initial  symptoms  are  diagnosed,  the 
problems  involve  similar  processes  of  making  tests  of  successively  smaller 
numbers  of  parts.  The  specific  locations,  test  instruments  and  values  at 
test  points  differ  for  different  problems,  but  the  same  hypothetico- 
deductive  process  is  used  to  narrow  and  isolate  the  problem  in  all  areas 
of  the  equipment  by  working  through  the  functional  dependencies.  Students 
learn  the  specifics  of  each  area  while  applying  the  common  hypothetico- 
deductive  process.  Thus,  the  learning  strategy  is  that  of  discovering 
specifics  In  the  functional  context  of  a  common  process. 

The  student  is  challenged  to  apply  the  common  process  and  to  learn  the 
specifics  in  the  problem  situation.  In  conventional  training,  as  a  rule 
specifics  are  first  learned  separately  and  then  applied.  This  convention 
is  contrary  to  the  functional  context  and  discovery  learning  research 
findings,  findings  which  have  largely  been  confined  to  the  laboratory. 
There  has  been  very  little  application  because  prior  to  the  advent  of  the 
low-cost,  random  access  machine,  discovery  learning  experiments  were  con¬ 
ducted  with  real  equipment  or  actual  equipment  trainers  (ATEs).  Research 
studies  could  garner  a  sufficient  amount  of  real  equipment  to  test  the 
learning  strategy,  but  schools  could  never  collect  enough  real  equipment 
to  Implement  the  strategy  In  practice. 

The  low-cost  microcomputer  makes  the  equipment  simulation  a  viable 
alternative  to  the  actual  equipment  as  a  learning  medium;  as  such  the 
present  effort  Is  designed  to  the  microcomputer  as  an  alternative.  This 
means  determining  whether  the  job  context  can  be  simulated  well  enough  for 
the  learning  to  take  place  in  the  problem-solving  situation  as  it  does  on 
the  job;  that  Is,  without  abstracting  the  job  specifics. 

The  term  Situational  Interaction  is  being  used  in  this  paper  to  refer 
to  the  type  of  learning  situation  created  by  the  present  simulation.  The 
interaction  takes  place  as  the  learner's  choice  of  response  to  one 
situation  results  In  being  required  to  confront  a  consequent  situation. 
Such  choices  and  consequent  situations  continue  until  the  learner  solves 
the  problem  imbedded  in  the  situatlon(s) .  The  choices  made  are  not 
correct  or  incorrect,  though  some  are  more  or  less  efficient  than  others 
In  solving  the  problem.  Repeated  trials  are  used  so  what  the  person 
learned  in  the  first  trial  can  be  applied  in  the  second.  In  the  second  or 
subsequent  trials,  more  Is  learned,  and  the  learner  becomes  more  efficient 
in  solving  the  problem.  A  series  of  such  Situational  Interaction  problems 
may  be  used,  with  each  simulating  some  aspect  of  the  real  situation,  to 
simulate  complex  jobs. 

The  technology  for  developing  Situational  Interaction  courseware  for 
low-cost  simulations  Is  not  generally  well  developed.  Techniques  have  not 
previously  been  developed  to  create  a  microcomputer  simulation  of  the  job 
situation  with  the  performance  requirements  imbedded  in  the  situation. 

The  techniques  are  quite  different  from  abstracting  the  requiranents  and 
then  using  a  simulator  as  the  medium  for  presenting  instruction. 


14 


This  paper  focuses  on  the  techniques  of  developing  simulations  with 
microcomputers  and  graphic  displays  that  imbed  the  job  requirements  in  the 
simulated  situation. 
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III.  DESCRIPTION  OF  THE  TRAINING  SYSTEM 


The  Site 


The  FB-111A  Flight  Simulator  at  Plattsburgh  Air  Force  Base  is  in  use 
24  hours  per  day.  It  is  used  in  pilot  training  for  two  shifts:  the  day 
shift  and  the  night  shift.  Maintenance  of  the  flight  simulator  takes 
place  on  the  midnight  or  midshift. 

While  flight  crews  are  using  the  Flight  Simulator  for  training,  main¬ 
tenance  personnel  are  on  standby  to  make  immediate  repairs  should  the 
simulator  break  down.  Generally,  the  most  highly  qualified  (five  level) 
personnel  are  assigned  to  this  duty.  Three-level  (apprentice)  personnel 
are  present  as  part  of  their  requirement  to  learn  on  the  job  by  assisting 
the  five-level  personnel.  Equipment  failures  are  usually  the  result  of 
normal  and  more  or  less  random  deterioration  of  circuit  components. 

If  the  maintenance  personnel  on  hand  cannot  repair  the  trouble  imme¬ 
diately,  it  is  written  up  as  a  maintenance  action  on  an  AFTO  Form  781  to 
be  addressed  on  the  night  shift. 

Access  to  the  equipment  by  research  personnel  was  generally  limited  to 
the  night  shift,  always  on  a  "no  interference"  basis.  R&D  activities  that 
required  Flight  Simulator  access  were  photographing  for  courseware  develop¬ 
ment,  verifying  symptoms  and  solutions,  and  performance  testing  for  the 
summative  evaluation. 

The  repair  and  tuning  of  the  simulator  for  use  by  the  flight  crews  is 
the  primary  mission  of  the  maintenance  personnel.  It  takes  precedence 
over  other  activities  including  training.  The  site  supervisor  is  reluc¬ 
tant  to  have  anything  done  to  the  flight  simulator  that  might  cause  it  not 
to  be  ready  later  in  the  day. 


The  Trainer/Simulator 


The  FSTT  is  a  graphic  display  simulator,  not  an  analogue  simulator. 
Analogue  training  devices  are  models  of  the  prime  equipment,  or  parts  of 
it,  constructed  to  generate  display  information  or  an  analogue  to  the 
parent  equipment  displays.  Early  electrical  analogues  modeled  components 
of  mechanical  systems.  This  approach  was  reasonable  because  the  equations 
of  motion  (harmonic)  for  mechanical  systems  are  so  similar  to  those  for 
electrical  systems  that  the  conversion  is  simple.  The  flight  simulator 
itself  uses  analogue  systems  to  mimic  the  performance  of  the  aircraft 
under  specific  but  quite  wide  ranges  of  operating  parameters. 

With  analogue  training  devices,  one  need  not  have  a  priori  knowledge  of 
the  equipment's  failure  modes  to  develop  troubleshooting  problems.  One 
merely  needs  to  "fail"  the  simulated  part  in  the  trainer,  and  all  the 
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resulting  effects  are  analogous  to  those  that  would  appear  in  the  prime 
equipment— depending,  of  course,  on  the  fidelity  of  the  model. 

In  the  FSTT  the  graphic  content  that  represents  the  prime  equipment's 
operations  is  produced  by  operating  the  prime  equipment  to  generate  the 
displays  and  taking  pictures  of  the  results.  The  content  depicted  on  the 

frames  of  the  fiche  is  taken  from  the  system  being  simulated.  For  the 

present  effort,  the  content  was  the  hardware  configuration  of  the  FB-111A 

Flight  Simulator  at  Plattsburgh  Air  Force  Base.  But  it  could  have  been 

any  type  of  prime  equipment.  Of  course,  new  courseware  content— i .e. , 
photographic  Images  of  the  prime  equipment  and  images  of  technical  data— 
have  to  be  developed  for  each  prime  system. 

The  FSTT  hardware  provides  random  access  to  selected  images;  and, 
thus,  the  hardware  is  generic  to  any  prime  equipment.  Only  the  images 
(courseware)  change  when  the  prime  equipment  changes.  The  hardware  for 
presenting  the  images  does  not  change. 

The  FSTT  is  shown  in  Figure  1  (see  the  following  page).  The  FSTT 
hardware  consists  of  dual,  random  access  microfiche  storage/display  units 
operated  via  hexidecimal  keypads,  a  magnetic  card  reader,  a  controller, 
and  a  printer.  The  logical  characteristics  are  defined  in  firmware 
(read-only  memory),  contained  within  a  microcomputer.  The  courseware  is 
stored  in  microfiche  cartridges  placed  in  slots  in  each  display  unit  and 
in  a  magnetic  card  that  holds  binary  data. 

The  controller  (a  microcomputer  using  an  Intel  80/24  microprocessor) 
handles  the  keyboard  Input,  the  selection  of  microfiche  frames,  the  pre¬ 
sentation  of  messages  on  the  in-line  display,  the  recording  of  student 
activities,  and  the  output  to  the  line  printer.  The  FSTT  is  a  stand¬ 
alone,  self-contained  system  and  requires  no  external  support  during  its 
regular  operation,  other  than  a  110-VAC  outlet. 

The  FSTT  was  constructed  and  assembled  in  accordance  with  the  best 
commercial  practices.  Its  design  and  fabrication  adhere  to  quality  con¬ 
structional  methods,  typical  of  laboratory  test  equipment.  It  functions 
reliably  In  an  environment  that  does  not  require  special  treatment  or  pre¬ 
paration  for  use  (60  to  100  degrees  Fahrenheit,  20  to  100  percent  relative 
humidity,  non-condensing).  When  not  in  use,  the  trainer  may  be  transported 
and  stored  In  a  harsher  environment  with  respect  to  temperature,  humidity, 
and  atmospheric  pressure. 

Major  components  of  the  FSTT  are  standard,  widely  available  commercial 
parts  with  low-cost  maintenance  and  replacement.  The  enclosure  (a  heavy- 
duty  metal  desk  with  provisions  for  holding  the  Internal  electronics)  is 
a  rigid  structure  with  hard  surfaces,  finished  to  provide  for  personnel 
safety,  protection  from  deterioration,  and  an  attractive  appearance.  The 
external  finish  is  light  blue,  with  a  walnut-colored  working  surface. 
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The  displays  (an  in-line  electro- luminescent  display  and  two  micro¬ 
fiche  storage/display  units)  provide  for  a  readable  size,  and  are 
conveniently  located  for  the  user.  The  keypads  are  simple  in  appearance 
and  use. 

No  unusual  or  specialized  equipment  is  required  to  test  or  maintain 
the  FSTT.  Diagnostics  leading  to  the  replacement  of  defective  major  com¬ 
ponents  (e.g.,  one  of  the  two  printed-circuit  cards)  can  be  performed  by 
means  of  internally  stored  diagnostic  procedures. 

The  principal  hardware  components  shown  in  Figure  1  are  these: 

Two  Microfiche  Storage/Display  Units  (1) 

Magnetic  Card  Reader  (2) 

Two  Hexidecimal  Keypads  (3) 

Electro- Luminescent  Display  (4) 

Quiet,  Hardcopy  Printer  (5) 

Microfiche  Cartridges  Inserted  in  Slots  (6) 

Electronic  Controller  (7),  consisting  of: 

Microprocessor  and  Memory  Card 

I/O  Interface  Card 

Card  Case,  Motherboard  and  Power  Supply 
Desk-Style  Enclosure 

Specific  Features 

A  description  of  the  specific  features  of  the  FSTT  follows.  The 
comercial  products  selected  for  this  application  do  not  represent  all 
possible  options;  they  are  the  alternatives  selected  by  the  researchers  in 
this  instance.  Their  selection  In  no  way  constitutes  an  endorsement  of 
the  products  by  the  United  States  Air  Force. 

Microfiche  Storage/Display  Units.  The  microfiche  storage/display 
components  are  Model  95s,  manufactured  by  Consolidated  Micrographics 
(formerly  Bruning  Division  of  AM  International,  Inc.).  These  are  auto¬ 
mated  retrieval /display  systems,  each  with  a  cartridge  holding  up  to  30 
microfiche.  The  left-hand  unit  is  Type  2498,  handling  fiche  of  7  x  14 
frame  format;  the  right-hand  unit  is  Type  2463  for  fiche  of  7  x  9  frame 
format.  Both  units  are  modified  so  that  the  FSTT  controller  and  its 
Interface  replace  the  keyboards  and  directly  drive  the  components  in  the 
selection  of  fiche  and  the  row/column  to  be  displayed.  The  left-hand  unit 
is  also  modified  to  provide  for  mechanical  support  only  of  the  in-line 
display  (DEI  Model  DE/432). 

Magnetic  Card  Reader.  The  magnetic  card  reader  is  the  Model  KB-31, 
manufactured  by  Vertel ,  Inc.  This  reader  is  interfaced  to  the  FSTT 
controller  via  a  parallel  input/output  (I/O)  port  on  the  microprocessor 
card. 
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Hex i decimal  Keypads.  Two  16-key  (4  x  4)  keypads  are  incorporated  into 
the  FSTT,  one  for  the  use  of  students  during  PRACTICE  and  EXAM  modes  of 
operation,  the  other  for  the  use  of  courseware  authors  in  the  AUTHOR  mode. 
The  two  keypads  are  identical  mechanically  and  electrically,  differing 
only  in  the  inscriptions  on  the  keys.  These  keypads  are  interfaced  to  the 
FSTT  controller  via  a  parallel  I/O  port  on  the  microprocessor  card. 

In-Line  Electro-Luminescent  Display.  The  in-line  display  is  the  Model 
DE/4'37,  manufactured  by  the  Digital  Electronics  Division,  Chemetrics,  Inc. 
This  display  provides  for  one  line  of  32  characters,  uses  a  5  x  7  dot 
matrix  for  each  character,  and  includes  character  encoders  for  the  full 
96-character  ASCII  set.  The  display  technology  used  is  vacuum  fluorescent 
with  a  blue-green  color,  filterable  to  blue,  green,  or  yellow. 

Printer.  The  hardcopy  printer  for  the  FSTT  is  the  Mini  term  Receive 
Only  ( RO )  Terminal  ,  Model  1201,  manufactured  by  Computer  Devices,  Inc. 

This  desk-top  printer,  is  operated  at  30  characters/second  and  supports 
the  entire  printable,  96-character  ASCII  set.  It  is  interfaced  in  the 
FSTT  to  the  serial  RS-232C  port  of  the  microprocessor  card,  within  the 
FSTT  controller. 

Electronic  Controller.  The  FSTT  controller  consists  of  three  prin- 
cipal  units:  the  microprocessor  card,  the  I/O  interface  card,  and  the 
supporting  card  cage,  motherboard,  and  power  supply.  The  microprocessor 
card  with  its  associated  memory  and  support  circuitry  is  the  iSBC  80/24, 
manufactured  by  Intel  Corporation. 

The  FSTT  I/O  interface  card  is  a  customized,  wire-wrapped  card  whose 
principal  function  is  to  condition  the  signals  sent  to  the  microfiche 
readers  by  the  iSRC  80/24  in  order  to  drive  the  fiche  carriers.  This  card 
also  provides  for  the  interface  to  the  in-line  alphanumeric  display. 

The  mechanical  and  electrical  support  for  the  controller  components  is 
provided  by  a  four-slot,  multibus  card  cage  and  motherboard  (Intel  iSBC 
Model  604):  by  a  power  supply  (Power  One,  Inc.,  Type  CP-291);  and  by  an 
electronic  chassis  assembly  (Vent  Rack,  Inc.,  Type  CH-A-7-X-21  with  Type 
TC-A-21  cover). 

Oesk  Enclosure.  The  entire  FSTT  is  contained  in.  and  mechanical ly 
supported  by  an  electronic  desk  enclosure.  Type  0D-266634-CL ,  manufactured 
by  Optima,  and  finished  in  blue/walnut.  This  is  a  heavy-duty,  metal  desk 
of  conventional  office  furniture  styling.  The  FSTT  controller  and  magne¬ 
tic  card  reader  are  contained  in  the  desk's  pedestal.  The  two  microfiche 
units  and  the  keypads  and  printer  are  supported  on  the  top  surface  of  this 
desk. 


The  Courseware 


FSTT  courseware  consists  of  individual  troubleshooting  problems  which 
are  identified  in  terms  of  an  observable  fault  and  which  contain  all  the 
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information  needed  to  identify  the  cause  of  malfunction,  make  the  needed 
repair,  and  verify  that  normal  operation  has  been  restored.  The  physical 
media  for  each  troubleshooting  problem  consisted  of  two  microfiche  and  one 
magnetic  card. 

Authoring.  The  courseware  in  this  effort  contained  30  typical  trouble- 
shooting  proolems  drawn  from  all  areas  of  the  prime  equipment  and  covering 
virtually  all  major  components.  Each  problem  was  to  be  solved  by  identi¬ 
fying  a  specific  failed  part  and  verifying  that  its  replacement  cleared 
the  symptoms  of  malfunction.  Part  failures  were  selected  from  actual 
malfunctions  that  had  occurred  on  the  job  in  operation  of  the  FB-111A 
flight  simulator.  The  maintenance  records  of  the  squadron  (380  AMS/MAAP ) 
were  examined,  and  a  list  of  failures  that  were  typical,  representative, 
and  challenging  was  developed.  The  records  provided  instances  where 
failure  symptoms  were  caused  by  a  single  failed  part,  by  two  failed  parts, 
and  by  no  failed  parts  but  a  misinterpretation  by  the  operator.  Trouble¬ 
shooting  problems  reflecting  all  three  types  were  constructed  as  follows: 


Problem  1 
Problem  2 
Problem  3 
Problem  4 
Problem  5 
Problem  7 
Problem  8 
Problem  9 
Problem  0 
Problem  10 
Problem  11 
Problem  12 

Problem  13 
Problem  14 

Problem  15 

Problem  16 
Problem  17 
Problem  18 
Problem  19 
Problem  20 
Problem  21 
Problem  22 
Problem  23 
Problem  24 
Problem  25 
Problem  26 
Problem  27 
Problem  28 
Problem  29 
Problem  30 


-  Hourglass  sweep  on  cockpit  Attack  Radar  (AR)  indicator 

-  Windshield  wiper  symptom  on  cockpit  AR  indicator 

-  Power  lost  (28  VDC) 

-  No  AR  video 

-  Incorrect  AR  video 

-  No  video  on  any  scope — no  time  signal 

-  Cross-hairs  do  not  fall  on  target 

-  Cannot  complete  fiducial  alignment  (oscillations) 

-  Mirror  image  on  AR  video 

-  Off-flag  on  Mach  indicator 

-  HSI  bearing  pointer  does  not  rotate 

-  Motion  “on"  lights  (platform)  remain  lit  after  motion 
has  been  deselected 

-  Number  1  UHF  does  not  transmit 

-  Instructor  console  light  does  not  light  when 
switchlight  is  depressed 

-  Instructor  console  does  not  show  AR  range  selected 
in  cockpit 

-  Sluggish  response  to  radical  stick  inputs 

-  Lagging  response  to  control  stick 

-  Main  OC  load  cannot  be  applied 

-  No  tanker  beacon  symbol  on  AR 

-  Antenna  tilt  not  within  limits 

-  Pilots  cannot  transmit  or  receive 

-  No  azimuth  marker  on  either  radar  set 

-  Instrument  Landing  System  (ILS)  identifier  inoperative 

-  Console  microphones  are  weak  and  garbled 

-  Terrain  Following  Radar  (TFR)  climb/dive  bar  inoperative 

-  Radar  Land  Mass  System  (RLMS)  power  supply  inoperative 

-  Navigator  cannot  communicate  with  pilot 

-  No  low  level  reflectance 

-  Right  shoulder  strap  locks 

-  Engine  performance  Indicators  do  not  correspond  with 
throttle  positions 
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A  description  of  each  of  the  problems  is  included  at  Appendix  A,  Aids  to 
Problem  Solution. 

Once  the  failures  were  selected,  it  was  necessary  to  verify  the  visible 
symptoms  and  signal  values  associated  with  each  failed  part.  These  effects 
could  be  predicted  to  some  degree  by  the  troubleshooting  expert  through  a 
knowledge  of  system  hardware  dependencies.  In  this  effort  the  actual 
equipment  was  also  used  to  establish  the  symptoms.  Either  the  failed  part 
was  placed  in  the  system  by  substituting  a  board  known  to  be  bad  from  the 
stock  of  such  boards  waiting  repair,  or  its  effects  were  Induced  through 
electronic  "jiggering";  e.g.  the  throwing  of  a  switch  or  complete  removal 
of  a  board.  Once  a  failure  had  been  inserted  or  induced  in  the  system, 
the  displays,  switches,  and  other  locations  that  would  be  examined  in  the 
process  of  troubleshooting  could  be  photographed  to  show  the  bad  symptoms 
and  test  point  readings. 

Courseware  was  made  up  of  four  types  of  materials  for  each  of  the  trouble¬ 
shooting  problems: 

1.  Photographic  simulations  of  the  physical  components  of  the 
prime  equipment  from  overall  views  of  the  cockpit,  instruc¬ 
tor  console,  and  cabinets,  down  to  individual  test  points. 

Where  a  physical  component  would  provide  a  visible  symptom, 
photographic  simulations  were  made  of  the  component's 
appearance  both  when  the  failure  symptom  was  present  and 
when  the  equipment  was  operating  correctly.  Where  an 
indication  of  a  good  or  bad  part  would  be  obtained  by 
reading  test  equipment  probing  a  test  point,  photographic 
simulations  were  made  of  the  test  equipment  display  both 

with  the  correct  value  (of  a  good  part)  and  the  incorrect 

value  (of  a  bad  part).  Verification  is  inherent  in  the 
troubleshooting  process  of  replacing  the  bad  component  and 
verifying  that  all  the  bad  outputs  are  now  good.  The 
courseware  author  must  take  pictures  of  the  good  outputs 

to  develop  the  verification  frames. 

2.  Facsimiles  of  technical  documentation  including  pages  from 
technical  orders,  block  diagrams,  schematics. 

3.  Lists  of  parts  that  could  be  replaced. 

4.  Lists  of  possible  actions  to  be  taken  next. 

The  photographic  simulations  comprised  the  bulk  of  the  courseware. 

They  were  produced  by  photographing  the  actual  equipment  as  troubleshooting 
steps  were  performed.  Pictures  were  taken  of  the  cockpit  with  all  its 
controls  and  displays.  Strategic  places  In  the  signal  path  between  the 
failed  part  and  the  final  output  were  identified  and  the  signal  measure¬ 
ments  photographed.  This  was  done  by  mounting  a  camera  on  the  test  instru¬ 
ment:  e.g.,  an  oscilloscope.  Pictures  were  taken  on  several  different 
routes  that  a  troubleshooter  might  reasonably  choose  to  isolate  the  fault. 
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Many  photographic  simulations  could  be  used  In  several  problems,  such 
as  pictures  taken  of  locations  of  parts;  e.g.,  cabinets  before  they  were 
opened,  when  they  were  open,  when  a  drawer  was  pulled  out.  Photographs  of 
"normal"  operating  signals  were  reused  from  problem  to  problem  because, 
aside  from  some  very  specific  signals  nrfiich  depend  on  switching  the 
setting  of  mode  and  range,  the  good  signals  are  generally  the  same  for  all 
problems. 

A  Mamlya  RB  67  Pro-S  camera  system  and  Tektronix  Oscilloscope  Camera 
were  used  to  support  the  data  collection  effort.  A  variety  of  films  and 
film-types  may  be  used.  Including  Polaroid  prints,  Polaroid  positive/ 
negative  film,  color  negative  film,  color  positive  film,  and  black-and- 
white  negative  film.  The  research  personnel  did  most  of  the  picture¬ 
taking,  however,  it  was  demonstrated  that  no  special  skills  were  needed. 
Figure  2  depicts  the  FSTT  courseware  authoring  process. 

A  full  description  of  the  courseware  authoring  techniques  is  included 
at  Appendix  B. 
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Figure  2.  FSTT  Courseware  Authoring  Process 

The  alternative  routes  to  solving  each  problem  were  mapped.  The 
routes  were  the  alternative  sets  of  steps  that  could  be  followed  to  per¬ 
form  the  troubleshooting.  This  phase  of  problem  authoring  can  be  easily 
accomplished  by  Individual  subject  matter  experts  using  their  trouble¬ 
shooting  competence  and  experience.  The  troubleshooting  strategy  for 
solving  the  problems  was  developed  with  the  the  technical  supervisory 
personnel  from  the  380  AMS/MAAD. 


After  following  several  troubleshooting  strategies  to  solution  and 
taking  pictures  for  each,  the  author  chooses  the  set  of  steps  that  would 
isolate  the  failed  part  using  the  fewest  number  of  checks.  This  is  called 
the  "optimum"  solution.  Other  troubleshooting  strategies  were  logical  but 
not  necessarily  the  most  efficient  for  the  specific  failure.  The  optimum 
path  is  much  easier  to  identify  after  completing  the  troubleshooting. 

Even  expert  troubleshooters  seldom  choose  the  optimum  path  when  they 
start--and  even  after  the  fact--not  all  experts  will  agree  on  the  best 
strategy.  The  optimum  path  is  relatively  efficient  with  respect  to  others. 

The  author  must  assign  the  selected  images  to  specific  locations  on 
the  two  fiche.  Each  of  these  locations  has  a  unique  two-digit  access  code 
so  that  when  this  code  is  entered  on  the  keypad,  the  corresponding  image 
is  displayed.  The  available  images  on  the  fiche  are  distributed  according 
to  preset  format  rules. 

The  locations  that  a  user  can  "go  to"  from  any  one  photograph  are 
given  their  two-character  access  codes.  These  codes  must  appear  over 
those  locations  on  that  photograph.  The  authoring  was  a  matter  of 
manually  putting  the  codes  on  the  pictures  and  putting  the  pictures  in  the 
appropriate  frames  of  the  two  microfiche  according  to  the  pre-established 
format. 

Pictures  of  technical  order  pages  that  are  relevant  to  the  steps 
described  in  the  troubleshooting  process  are  likewise  given  access  codes. 
For  instance,  there  are  codes  that  enable  users  to  go  to  detailed  schema¬ 
tic  diagrams  selected  from  overall  diagrams.  As  a  check  on  the  codes  and 
pictures,  the  author  goes  through  each  troubleshooting  path,  checking  to 
ensure  that  the  records  codes  refer  the  user  to  the  Intended  steps.  The 
author  then  uses  a  layout  that  graphically  displays  the  locations  for  each 
picture  on  the  microfiche. 

The  coded  pictures,  identified  as  to  which  frame  of  each  microfiche 
they  are  to  be  placed  on,  are  sent  to  the  microfiche  printer.  The 
troubleshooting  problems  developed  in  this  effort  had  one  color  and  one 
black-and-white  microfiche.  Pages  from  TOs  are  placed  on  the  black-and- 
white  fiche,  as  are  test  point  readings.  Pictures  of  the  equipment  are 
placed  on  the  color  fiche.  The  black-and-white  fiche  require  2  to  3  days 
for  film  processing;  the  color  fiche  require  about  2  weeks.  The  cost  was 
less  than  a  dollar  a  copy  for  the  black-and-white  fiche  a  few  dollars  for 
color.  The  two  fiche  are  loaded  into  cartridges,  one  for  each  viewer. 
Usually  about  15  problems  were  loaded  into  the  cartridges  at  one  time. 

Scoring.  The  scoring  system  provided  extrinsic  feedback  to  the  student 
on  how  good  the  choices  were.  The  system  was  developed  from  what  the 
experts  (on  the  average)  said  was  the  most  efficient  path  to  take. 

With  the  fiche  and  location  records  in  hand,  the  author  entered  the 
scoring  codes  on  a  plastic  magnetic  card.  The  plastic  card  was  placed  in 
the  magcard  reader  slot  and  the  author's  keypad  was  used  to  enter  the 
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efficiency  scores  for  each  choice.  This  required  only  a  few  hours  to 
accomplish.  The  program  entered  on  the  plastic  card  was  scoring  for  the 
paths  used  to  solve  the  problem  and  also  instruction  to  the  printer  for 
recording  data  on  use  (i.e.,  problem  number,  student  ID  number,  time, 
data,  and  each  access  code  entered  on  the  hexi decimal  pad). 

The  original  system  design  provided  for  performance  judgments  to  be 
displayed  after  each  action  taken  by  the  student.  Extrinsic  feedback  was 
displayed  on  the  monitor  in  terms  of  the  efficiency  of  the  action  in 
finding  the  most  direct  route  to  the  faulty  part.  The  most  efficient 
action  possible  received  three  pluses;  the  least  efficient  action  received 
three  minuses.  The  plus  and  minus  messages  were  reinforced  verbally  with 
"relevant"  and  "Irrelevant"  messages  on  the  in-line  display.  In  addition, 
the  capability  was  built  into  the  system  to  provide  routine  CAI-type  feed¬ 
back  of  "correct"  and  "incorrect"  responses  in  the  event  a  courseware 
author  wanted  to  Include  CAI-type  verbal  questions  in  the  sequences. 

Situational  Interaction  does  not  employ  CAI  verbal  strategies.  The 
situation  provides  the  feedback  on  the  correctness  of  the  solution;  i.e., 
when  the  faulty  part  Is  replaced,  the  situations  displayed  to  the  student 
change  from  malfunction  symptoms  to  evidence  of  correctly  functioning 
equipment.  The  student  must  simulate  job  actions  to  determine  that  the 
malfunction  has  cleared.  The  student  does  this  by  performing  the  series 
of  troubleshooting  actions  again  and  verifying  that  the  bad  test  readings 
and  other  symptoms  are  gone. 

The  system  was  also  capable  of  having  the  in-line  display  feedback 
appear  intermittently  according  to  various  reinforcement  schedules. 
Reinforcement  is  effective  In  making  a  single  response  resistant  to 
extinction;  however,  this  is  antithetical  to  the  objective  of  learning  to 
solve  problems.  Therefore,  It  was  illogical  to  include  modes  for  reinfor¬ 
cement  schedules  in  the  FSTT  hardware  design.  This  fact  was  amusingly 
demonstrated  vrfien  some  users  accidentally  set  the  FSTT  on  a  reinforcement 
mode;  they  perceived  that  the  FSTT  was  malfunctioning  because  it  did  not 
display  the  efficient  score  each  time  the  user  made  a  choice. 

Job  Aids.  When  the  FSTT  was  first  delivered  to  Plattsburgh  AFB,  the 
job  supervisors  saw  that  it  could  be  used  on  the  job  as  an  aid,  as  well  as 
for  training  before  the  job  is  performed.  The  researchers  explained  that 
use  as  a  job  aid  would  require  the  development  of  a  "map"  having  routes 
that  went  to  each  replaceable  part.  The  routes  in  the  map  used  for 
training  go  only  to  the  replaceable  parts  that  were  selected  as  faults 
for  the  problems.  The  AFHRL  deemed  such  a  job-aiding  tool  worth  investi¬ 
gation.  One  area  of  the  flight  simulator  (Landmass)  was  selected  for  this 
development.  The  contract  was  modified  to  produce  the  job-aiding  sample 
on  the  Landmass  subsystem.  When  it  was  time  to  test  this  sample  job- 
aiding  material,  the  Air  Force  job  supervisor  had  second  thoughts  about 
going  ahead  with  the  test.  Because  Air  Force  regulations  specify  that 
only  approved  technical  documentation  may  be  used  for  maintenance  on 


?6 


operational  equipment,  it  was  decided  that  use  of such  non-regulation 
materials  to  support  actual  job  performance — as  opposed  to  training— would 
be  a  violation  of  regulations. 

The  job  aiding  materials  that  were  developed  have  the  following 
characteristics: 

1.  The  routes  to  each  replaceable  part  (in  the  Landmass 
subsystem)  are  identified  and  "mapped"  just  as  the  routes 
to  the  selected  malfunction  are  mapped  with  the  standard 
problems. 

2.  The  user  is  free  to  follow  any  route  to  solution. 

3.  The  map  of  the  routes  is  based  on  a  Hierarchically 
Structured  Functional  Organization  (HSFO). 

The  routes  to  each  replaceable  part  are  developed  in  the  same  manner 
as  the  routes  to  a  specific  part  are  developed.  That  is,  each  replaceable 
part  is  considered  in  turn.  The  initial  symptom  of  its  malfunction  Is 
identified  and  pictures  are  taken:  other  test  points  are  Identified  on  the 
route  between  the  symptom  and  the  part,  and  those  pictures  are  taken.  In 
this  process,  it  should  be  recognized  that  there  is  substantial  redun¬ 
dancy.  That  is,  the  initial  portion  of  routes  are  common  to  several 
replaceable  parts.  The  middle  section  of  routes  are  common  to  fewer,  but 
usually  more  than  one  part,  and  only  the  final  tests  are  unique  to  one 
replaceable  part. 

The  hierarchical  structure  of  the  functional  components  is  this 
branching  of  routes  from  initial  symptom  to  replaceable  parts.  The  orga¬ 
nization  of  this  hierarchical  structure  may  be  learned,  or  it  may  be  used 
as  an  aid  on  the  job.  Using  this  type  of  job  aid  represents  practice. 

The  entire  hierarchical  structure  may  be  learned.  It  should  be  noted  that 
this  is  not  the  case  with  Job  Performance  Aids  (JPAs).  The  developers  of 
JPAs  construct  the  "trees"  or  hierarchical  structure  of  the  system,  but 
the  JPA  (in  all  its  forms)  does  not  make  this  structure  evident  to  the 
user.  The  user  follows  the  steps  given  in  the  JPA  and  may  memorize  certain 
steps  or  sequences  of  steps,  but  typically  does  not  perceive  the 
hierarchical  structure. 

The  job  aiding  materials  produced  in  the  present  effort  were  Intention¬ 
ally  designed  to  reveal  the  organization  of  the  functional  entitles  in  the 
hierarchical  structure.  This  represents  new  RAD  not  merely  a  new  applica¬ 
tion  of  JPAs. 

To  use  the  job-aiding  materials,  the  user  first  looks  at  the  symptom 
of  malfunction  on  the  actual  equipment.  Then,  via  the  FSTT,  the  user 
accesses  the  symptom,  which  is  broken  into  optional  choices  in  the  job- 
aiding  materials.  The  user  chooses  the  feature  of  the  symptom  that  is 
like  the  one  seen  on  the  actual  equipment,  enters  the  choice  on  the  FSTT 
keypad,  id  receives  a  view  of  the  situation  resulting  from  that  choice. 
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This  situation  also  has  choices.  The  user  makes  test  measurements  on  the 
actual  equipment  and  selects  the  choice  where  a  bad  test  point  reading  was 
found.  After  entering  this  choice  in  the  keypad,  the  user  proceeds  in  a 
like  manner  until  the  (bad)  replaceable  part  is  identified.  That  part  is 
then  replaced  and  the  result  verified. 

The  user  may  “back-up"  on  the  FSTT  at  any  time  or  start  over.  As  with 
the  FSTT  problems,  the  user  learns  the  hierarchical  structure  while  moving 
down  (and  back)  the  routes  to  the  various  sources  of  trouble. 

It  is  recognized  that  the  job-aiding  materials  can  be  used  for  training 
as  well  as  in  support  of  actual  job  performance.  That  is,  users  can  move 
through  the  routes  in  the  hierarchical  structure  in  order  to  learn  the 
structure.  However  it  appears  that  making  problems  out  of  job-aiding 
material  would  provide  a  better  learning  situation.  Once  the  job-aiding 
materials  have  been  developed,  most  of  the  development  work  for  making 
problems  has  been  done.  So  conversion  to  problems  would  be  an  easy  matter. 
This  was  not  done  in  the  present  effort  because  it  was  anticipated  that 
the  job-aiding  mateials  would  be  tested  on  the  operational  equipment. 


Using  the  FSTT 

When  selecting  a  problem  to  work  on,  the  user  inserts  the  plastic  card 
for  that  problem  In  the  card  reader.  This  causes  one  fiche  to  be  drawn 
from  each  of  two  cartridges  mounted  below  the  two  display  screens.  Most 
problems  have  about  100  pictures  on  each  of  two  fiche. 

To  Initiate  the  problem,  the  user  enters  "00"  on  the  keypad.  The  user 
then  sees  a  description  of  the  symptom  of  the  problem  on  one  screen  and  a 
line-drawing  overview  of  the  parent  system  on  the  other  screen.  The 
symptom  is  described  on  Air  Force  Form  781  as  it  would  be  prepared  by  the 
person  who  first  noted  the  problem  on  the  operational  equipment.  This  is 
the  usual  way  a  maintenance  person  receives  notification  of  the  problem  in 
the  job  situation. 

The  user's  first  step  is  to  look  at  the  symptom.  Overlaid  on  the 
overview  of  the  equipment  are  a  number  of  alphanumeric  access  codes  at  the 
locations  of  the  major  components  of  the  system.  These  are  the  choices  of 
where  to  look  for  the  symptom.  The  user  must  choose  one  of  the  coded 
locations.  Entering  the  code  for  this  choice  on  the  keypad  causes  a 
picture  of  the  major  component  to  appear  on  the  screen  where  the  Form  781 
was  previously  shown.  Coded  locations  on  that  picture  allow  the  user  to 
call  up  detailed  views  of  the  displays  visible  or  available  at  that  loca¬ 
tion.  There  are  usually  several  displays  from  which  to  choose.  The  user 
must  recognize  the  specific  display  that  will  show  the  symptom  and  enter 
its  access  code  on  the  keypad.  A  detailed  image  of  this  display  then 
appears  on  the  other  screen. 

The  frame  from  which  the  access  code  was  selected  always  remains  on 
its  screen  while  the  newly  accessed  frame  appears  on  the  screen  of  the 
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alternate  display  unit.  These  two  related  views  remain  until  a  new  frame 
is  selected  from  among  the  codes  shown  on  the  images.  An  access  code  on 
the  second  frame  can  be  entered,  and  a  new  image  will  replace  the  original 
image  on  the  first  viewer.  Or  an  alternative  code  can  be  selected  off  the 
first  image,  and  a  different  frame  will  appear  on  the  second  screen. 


The  user  studies  the  symptom  display  to  formulate  a  first  hypothesis 
as  to  the  source  of  the  problem.  The  user  deduces  from  this  hypothesis 
what  other  symptoms  or  test  point  readings  will  be  bad  and  must  then  look 
at  these  symptoms/readings  to  confirm  or  deny  the  hypothesis. 


The  overview  of  the  equipment  will  give  the  user  the  location  codes 
for  collecting  the  needed  malfunction  data.  Several  kinds  of  information 
are  available.  The  alphanumeric  codes  on  the  Images  are  shown  inside 
circles,  diamonds,  squares,  or  triangles.  These  shapes  identify  the  type 
of  information  the  code  will  retrieve. 


The  circle  refers  to  an  observation  location.  When  a  circled 
code  is  entered,  the  user  will  get  a  view  of  what  could  be  observed  at  the 
location  marked  by  the  circle.  The  observation  may  be  a  more  detailed 
view  (e.g.,  one  cabinet  enlarged  from  many  cabinets  or  one  area  of  focus 
on  a  large  panel)  from  which  the  user  can  recognize  the  symptom,  or  it  may 
be  a  view  of  what  would  appear  on  the  test  Instrument  when  a  probe  is 
applied  to  that  location  (e.g.,  the  output  read  at  a  test  point). 


If  the  code  is  in  a  square  u  ,  choosing  It  will  provide  a  view  of  a 
block  diagram,  schematic  diagram,  or  other  information  found  on  a  page 
from  a  TO.  The  content  of  the  TO  page  will  relate  to  what  the  user  is 
physically  observing  on  the  other  screen.  The  page  from  the  TO  has  codes 
on  it  for  retrieving  more  detailed  views  of  schematics. 


Choosing  a  code  in  a  triangle  A  provides  a  menu  of  optional 
replacement  actions  that  can  be  taken. 

If  the  code  is  in  a  diamond  o,  the  user  will  receive  a  strategy 
summary,  such  as  a  list  of  optional  actions  that  are  reasonable  to  try  at 
this  point.  This  form  of  help  is  available  only  in  the  training  mode. 


At  any  time,  the  user  may  go  back  to  a  previous  display  (up  to  five 
steps  back)  or  start  from  the  beginning.  The  user  may  choose  an  access 
code  of  a  test  point  and  obtain  a  picture  of  the  test  Instrument  showing 
the  reading  at  that  point.  The  user  may  also  choose  access  codes  that 
provide  schematics  or  other  information  from  TOs,  and  then  choose  access 
codes  on  schematics  to  obtain  other  schematics  with  more  detail  on  the 
component  of  Interest.  When  finished  collecting  information  and  ready  to 
make  a  replacement,  the  user  chooses  an  access  code  that  provides  a  "menu" 
of  possible  replacement  actions.  Among  the  options  are  correct  replace¬ 
ments  which  are  safe  as  well  as  some  replacement  options  that  are  unsafe. 

A  user  who  decides  to  make  a  replacement  that  is  both  incorrect  and 
dangerous  gets  an  "abort"  message  on  the  in-line  display  and  must  start 
over. 
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Before  choosing  an  action  the  user  may  want  to  review  the  replacement 
procedures  in  the  TO.  This  is  done  by  selecting  the  access  code  for  the 
replacement  action  being  considered.  If  the  procedure  in  the  TO  indicates 
that  the  action  is  appropriate,  the  user  then  enters  the  code  for  replacing 
that  part  and  verifies  that  replacing  the  part  has  cleared  the  symptoms. 

To  confirm  the  correct  repair  has  been  made,  the  user  must  go  back  to 
reexamine  the  test  point  readings  and  symptoms  that  were  bad.  In  fact, 
the  entry  of  the  correct  replacement  code  changes  all  the  frames  to 
correct  test  point  readings  and  good  symptoms,  but  this  is  not  confirmed 
until  the  student  checks  it.  If  the  faulty  part  is  replaced,  all  these 
readings  and  symptoms  are  now  good.  If  the  problem  has  more  than  one 
malfunctioning  part,  the  symptoms  for  the  part  correctly  replaced  will  be 
good,  but  bad  symptoms  will  remain  for  the  other  faulty  part. 

After  deciding  the  repair  has  been  verified,  the  user  presses  a  CC 
code  on  the  hexidecimal  pad  and  receives  a  performance  score  on  the  in¬ 
line  display.  At  this  point,  the  display  will  tell  whether  or  not  the 
problem  has  been  solved.  It  must  be  remembered  that  the  scoring  system 
for  the  FSTT  is  oriented  to  the  problem  at  hand.  The  highest  score  is 
given  to  the  performance  with  the  fewest  frame  requests--those  that  are 
directly  relevant  to  solving  that  particular  problem.  A  complete  record 
of  all  the  user's  actions  Is  printed  out  on  the  hard  copy  printer. 

In  a  few  problems  (called  zero  problems),  a  symptom  will  not  be  seen. 
The  user's  objective  on  zero  problems  is  to  recognize  that  the  write-up  on 
the  Form  781  work  order  was  Incorrect. 

Each  problem  provides  options  that  allow  the  user  to  create  alter¬ 
native  paths  for  collecting  Information  about  the  pattern  of  symptoms  and 
test  point  readings  produced  by  the  malfunction  which  caused  the  initial 
symptom  of  failure. 

Some  paths  are  more  efficient  than  others.  Some  paths  lead  only  to 
good  readings  and  thus  become  dead  ends.  That  is,  a  good  signal  means 
that  components  providing  inputs  to  it  are  also  good.  If  a  path  with  only 
good  readings  is  taken,  the  user  is  "off"  the  path  leading  to  the  bad 
component. 

The  readings  and  symptoms  represent  Intrinsic  feedback,  the  same  kind 
of  feedback  the  parent  system  provides  when  a  technician  makes  the  checks 
and  takes  the  actions  the  student  is  doing  on  the  simulated  equipment. 

The  user  receives  another  kind  of  feedback  in  the  training  mode.  An  in¬ 

line  display  above  the  left  screen  gives  the  student  a  judgment  of  the 
efficiency  of  each  step  taken  (on  a  six-point  scale)  and  a  cumulative 
score  on  how  efficient  the  selection  of  steps  has  been  on  the  average. 
Efficiency  does  not  equal  correctness.  The  user  can  correctly  identify 
and  repair  a  fail ure  with  any  degree  of  efficiency.  In  the  test  or  pure 

simulation  mode,  the  user  is  limited  to  Intrinsic  feedback  only.  Some 

supervisors  preferred  to  keep  the  FSTT  in  the  pure  simulation  mode  at  all 
times. 
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While  looking  for  the  bad  indications,  the  user  is  collecting  informa¬ 
tion.  Even  if  the  indication  is  good,  that  finding  is  useful  information. 
Getting  a  good  indication  will  require  the  user  to  modify  the  working 
hypothesis.  From  the  modified  hypothesis  a  new  deduction  about  the  loca¬ 
tion  of  a  bad  reading  is  made  and  checked.  Several  good  indications  in 
turn  will  suggest  that  the  hypotheses  are  not  very  effective. 

Sometimes  choices  are  equally  likely  and  sometimes  not.  The  most 
efficient  set  of  choices  is  where  each  one  reveals  a  bad  symptom  or 
reading.  Such  a  set  usually  requires  a  little  luck  in  choosing  between 
equally  likely  probabilities.  Some  problems  require  more  choices  than 
others  to  solve  the  problem.  The  user  must  modify  a  hypothesis  until  each 
specific  deduction  from  it  identifies  a  location  that  has  a  bad  symptom  or 
test  point  reading. 

In  this  simulated  experience,  the  user  abstracts  information  from  each 
situation  depicted  on  a  screen,  then  uses  the  information  to  restructure  a 
hypothesis  and  choose  the  next  situation.  The  next  situation  may  or  may 
not  be  what  was  expected.  Regardless,  the  user  must  get  out  of  that 
situation  and  into  another.  The  user  interacts  with  each  situation.  The 

situation  does  not  tell  the  user  what  to  do  next.  The  user's  hypotheses 
and  deductions  structure  the  information  that  is  abstracted  and  influences 
the  choice  of  what  to  do.  This  is  similar  to  the  actual  job  experience 
and  is  called  synthetic  experience.  When  working  on  this  problem  the  next 
time,  the  student  will  have  in  mind  the  structure  derived  from  the  hypo¬ 
theses  and  experience  from  the  first  trial.  This  cognitive  understanding 
is  quite  different  from  learning  a  pre-established  sequence  of  steps  to 
solve  each  problem. 

One  sample  problem  was  made  to  acquaint  users  with  the  FSTT.  This 
problem  is  included  at  Appendix  C. 


FSTT  Cost  Effectiveness  Evaluation 


FSTT  hardware  and  software  costs  Include  construction  and  development 
costs.  To  exclude  the  development  costs,  the  costs  of  the  components  and 
assembly  have  been  broken  out  of  the  total  costs.  FSTT  hardware  components 
amounted  to  $15,000.  The  two  microfiche  viewers  accounted  for  $7,000  of 
this  total.  The  cost  of  assembling  the  components  was  approximately 
$10,000.  The  cost  of  components  and  assembly  would  have  been  less  for 
quantity  production,  depending  on  the  kind  of  assembly  line  process  which 
was  established  for  the  quantity  ordered.  For  an  order  of  hundreds  of 
devices,  the  cost  probably  would  have  been  on  the  order  of  $15,000  for  each 
copy. 

The  life  cycle  costs  of  the  device  are  essentially  maintenance  costs 
and  an  amortization  of  the  capital  cost  over  the  device’s  life.  The  main¬ 
tenance  costs  for  the  year  the  system  was  at  the  job  site  included  the  cost 
of  correcting  design  bugs  as  well  as  of  routine  maintenance.  The  life  of 


the  system  can  be  estimated  from  one  year  of  wear  and  tear.  We  estimate 
the  hardware  will  survive  for  three  to  five  years  on  a  job  site.  Routine 
maintenance  costs  for  the  hardware  amounted  to  approximately  $1,300  during 
one  year.  This  amount  would  increase  in  subsequent  years,  probably 
reaching  $3,000  in  three  years  and  $5,000  in  five  years. 

The  life  cycle  for  three  years  would  be  as  follows: 

Three-Year  Life  Cycle 


MAINTENANCE  - 


$15,000  *  3  years  =  $5 ,000/year 


1st  Year  $1,300 
2nd  Year  2,000 
3rd  Year  3,000 


$6,300 


$6,300  f  3  years 
TOTAL/YEAR 


=-$2*499/yeap- 
$7, 100 /year 


THREE-YEAR  LIFE  =  3  x  $7,100  «  $21,300 
(No  Resale  Value) 


Five-Year  Life  Cycle 


COST  = 

MAINTENANCE  - 


$15,000  «•  5  years  =  $3 ,000 /year 


1st  Year  $1,300 
2nd  Year  2,000 
3rd  Year  3,000 
4th  Year  4,000 
5th  Year  5,000 


$15,300 


$15,300  *  5  years  =  $  3,060/year 


TOTAL /YEAR 


$  6,060/year 


FIVE-YEAR  LIFE  =  5  x  $6,060  =  $30,300 

(No  Resale  Value) 


Depending  upon  the  number  of  years  of  amortization,  assumptions  about 
the  number  of  copies  of  the  hardware  ordered,  and  failure  rates  over  a 
period  of  3.5  years,  the  total  life  cycle  system  costs  are  on  the  order  of 
six  to  seven  thousand  dollars  per  year  or  about  $21,300  for  a  three-year 
life  and  and  $30,300  for  a  five-year  life. 


The  cost  of  alternative  hardware,  in  the  form  of  videodisc  player  and 
microprocessor,  would  be  less  than  $5,000  for  the  same  hardware  capability. 
At  the  time  the  FSTT  hardware  was  designed,  the  videodisc  mastering  process 
was  not  reliable.  However,  it  is  reliable  at  the  time  of  this  report.  A 
videodisc-based  system  has  one  inherent  disadvantage  a  photographic  system 
does  not  have.  The  schematic  diagrams  in  the  technical  documentation  have 
horizontal  lines  between  parts  that  will  not  be  shown  through  the  roster 
lines  on  a  standard  cathode  ray  tube.  Presumably  this  problem  can  be  over¬ 
come  by  using  the  video  camera  to  make  close  up  views  of  schematic  diagrams 
after  showing  a  view  of  the  whole  schematic  page.  This  close-up  procedure 
was  not  tested  in  the  current  effort.  Although  there  is  no  experimental 
evidence  of  the  effectiveness  of  the  procedure,  it  is  reasonable  to  believe 
that  this  general  capability  of  video  cameras  to  "blow  up"  an  image  would 
be  as  effective  for  schematic  diagrams  as  it  is  for  other  images. 

A  videodisc-based  system  has: 

1.  the  same  capability  for  selecting  and  presenting  the  images 
of  the  FSTT  courseware,  in  color  and  equal  visual  quality 
with  the  same  response  time  as  the  microfiche  device, 
assuming  the  schematic  diagram  problem  is  readily  solved 
by  a  close-up  view. 

?.  a  cost  that  is  less  than  one  third  the  cost  of  the  micro¬ 
fiche-based  system.  The  total  cost  of  the  videodisc-based 
system  is  less  than  the  cost  per  year  of  the  microfiche- 
based  system. 

3.  reliability  that  is  inherently  greater  because  it  has  fewer 
moving  parts  than  the  microfiche  system. 

The  FSTT  courseware  and  software  were  the  ingredients  producing  the 
result  in  the  current  effort.  The  content  of  the  situational ly  interactive 
FSTT  training  problems  can  be  converted  to  videodisc  images  merely  by  using 
a  video  camera  on  the  image  content.  The  software  of  the  FSTT  device  can 
he  programmed  on  a  microprocessor  to  cause  the  images  on  videodisc  to  unfold 
in  the  same  manner  they  unfolded  to  student  action  on  the  FSTT  device.  In 
short,  a  videodisc-based  system  can  support  the  situational ly  interactive 
content  that  produced  the  results  in  this  effort.  Therefore,  a  videodisc 
system  and  a  microprocessor  added  to  the  videodisc  player  can  be  programmed 
to  duplicate  the  functions  of  the  microprocessor  in  the  microfiche-based 
system  and  would  be  more  cost  effective  than  the  microfiche-based  system 
used  in  the  FSTT  experiment. 


IV.  FORMATIVE  EVALUATION 


Evaluation  Plan 


In  general,  formative  evaluation  is  the  developmental  stage  of  trying 
out  an  RAD  product  and  revising  it  on  the  basis  of  observations  made 
during  its  use.  The  Formative  Evaluation  Plan  for  the  FSTT  called  for  the 
research  staff  to  deliver  the  system  to  the  job  site  at  Plattsburgh  AFB 
(380  AMS/MAAD)  and  remain  there  for  2  weeks  to  help  set  it  up,  demonstrate 
its  use,  and  show  job  supervisors  how  to  develop  courseware  problems  them¬ 
selves.  The  FSTT  was  placed  near  the  operational  flight  simulator  and  was 
available  for  use  24  hours  each  day.  Personnel  who  were  not  on  duty 
during  a  shift  could  use  the  FSTT;  and  if  personnel  on  a  work  shift  were 
not  too  busy,  they  could  practice  on  it.  One  civilian  instructor  was  very 
enthusiastic  about  the  usefulness  of  the  FSTT  and  was  made  the  chief  point 
of  contact  during  formative  evaluation.  In  that  role,  the  instructor  pro¬ 
vided  initial  introduction  of  the  FSTT  to  users  and  was  often  available  to 
discuss  problems  with  them.  This  instructor  also  collected  and  sent  the 
hard  copy  printout  data  on  trainee  performance  to  Kinton  for  analysis. 

The  purposes  of  the  evaluation  were  to  ensure  that  the  courseware  was 
technically  correct,  the  system  was  accepted  by  supervisors  and  trainee 
users,  the  hardware  was  reliable  and  free  of  design  flaws,  and  competent 
troubleshooters  could  develop  courseware  with  ease.  Sources  of  evaluation 
data  were  observations  made  by  research  personnel,  print-out  records  of 
the  FSTT  itself,  and  questionnaires  answered  by  job  site  personnel.  The 
products  of  the  formative  evaluation  were  to  be:  perfected  and  reliable 
hardware,  verified  and  validated  courseware,  user  acceptance  measurements, 
and  user  performance  data. 

According  to  the  evaluation  plan  all  hardware  failures  were  to  be 
corrected  promptly  by  Kinton.  If  the  failure  was  due  to  a  design  fault, 
the  design  was  to  be  revised.  If  it  was  caused  by  something  that  could  be 
prevented,  a  preventive  maintenance  procedure  was  to  be  developed.  If  the 
fault  was  caused  by  normal  wear,  it  was  to  be  noted. 

The  courseware  was  to  be  evaluated  in  two  increments:  20  problems 
delivered  with  the  hardware  and  10  developed  at  a  later  date.  After  the 
job  supervisors  had  examined  and  recommended  revisions  to  the  first  20 
problems,  the  second  increment  was  to  be  developed  consistent  with  the 
recommended  revisions.  As  revisions  to  the  first  20  problems  were  being 
made,  the  reamlning  problems  were  being  used  by  on  site  personnel  — 
inexperienced  and  experienced  alike.  During  the  period  of  April  through 
November  1982,  the  number  of  problems  available  for  users  increased  from  5 
to  30  as  revisions  were  made  and  new  problems  developed. 


Results 


Hardware.  After  its  delivery  to  Plattsburgh  AFB,  the  FSTT  was 
generally  reliable  but  had  some  hardware  failures.  Most  were  corrected 
within  days  or  weeks,  but  one  intermittent  failure,  associated  with  the 
magcard  reader,  resulted  In  a  several -month  shutdown.  The  source  of  dif¬ 
ficulty  was  that  a  longer  data  cable  for  connecting  the  card  reader  to  the 
computer  was  substituted  for  the  one  supplied  by  the  card  reader  manufac¬ 
turer.  The  added  length  resulted  in  lower  signal  strength  and  intermittent 
inability  of  the  computer  to  get  the  data  needed  for  processing.  The  FSTT 
design  was  changed  to  provide  for  a  cable  of  proper  length  between  the 
card  reader  and  the  computer. 

At  one  point  the  FSTT  failed  to  accept  the  calibration  data  or  the 
problem  card  data.  A  head  cleaning  card  had  been  provided  to  remove  accumu¬ 
lations  of  magnetic  particles  on  the  head;  however,  job  site  personnel  had 
not  performed  this  preventive  maintenance,  and  the  head-cleaning  card  pro¬ 
vided  for  this  purpose  could  not  be  found.  When  the  head  was  cleaned, 
normal  operation  was  resumed. 

Originally,  users  reported  they  could  not  tell  If  the  FSTT  was  working 
when  first  turned  on.  In  order  to  provide  some  feedback  to  the  user,  a 
software  change  was  written  so  that  the  in-line  display  showed  an  "R"  when 
the  data  card  was  being  read,  a  "W"  when  being  written  to,  and  a  "D"  when 
outputting  to  the  printer.  In  addition,  if  the  card  reader  was  unable  to 
accept  valid  data  from  the  card,  one  of  the  set  of  error  messages  was 
displayed  (e.g.,  “try  another  track"). 

Other  hardware  problems  were  of  a  minor  nature.  They  are  reported  in 
Appendix  D. 

Courseware.  The  original  courseware  troubleshooting  problems  were 
examined  In  detail  by  job  site  personnel  who  had  many  years  of  experience 
maintaining  the  flight  simulator.  They  worked  through  each  problem.  Some 
courseware  content,  usually  a  photograph,  but  sometimes  a  diagram,  required 
correction.  Of  the  20  problems,  15  needed  some  revision. 

Corrections  to  the  problems  entailed  selecting  substitute  illustra¬ 
tions,  coding  them,  rephotographing ,  and  resubmitting  the  package  to  the 
micropublisher  for  reshooting  Into  microfiche.  The  revised  fiche  were 
mailed  to  Plattsburgh  AFB  for  Inspection  and  insertion  into  the  cartridges. 
No  more  than  two  revisions  were  ever  needed  for  any  problem.  The  revisions 
were  simple  and  Inexpensive  to  make  on  microfiche.  For  the  second  set  of 
10  problems  the  job  supervisors  suggested  that  the  problems  be  "easier" 
because  the  first  set  had  a  sufficient  number  of  difficult  problems.  In 
effect  "easier"  meant  problems  for  which  there  were  only  a  few  possible 
causes  of  the  bad  symptoms  and  which  the  student  could  solve  by  using  pro¬ 
bability  of  failure  to  replace  components,  without  obtaining  data  from 
additional  test  points.  No  requests  were  made  for  additional  multiple 
malfunction  problems- -which  are  the  most  difficult  of  all. 
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Site  experts  believed  that  scoring  of  a  few  problems  should  be 
revised.  This  was  done  by  recoding  the  magnetic  card  and  did  not  require 
revision  of  the  fiche.  The  magcards  were  revised  the  same  way  they  were 
originally  made;  i.e.,  the  card  for  a  problem  was  put  in  the  FSTT  card 
reader  slot,  and  the  scoring  codes  were  entered  in  Author  mode  and 
recorded  on  all  four  stripes  of  the  magcard.  Changing  the  scoring  on  a 
magcard  took  no  more  than  1  or  2  hours  per  problem. 

User  Acceptance.  User  acceptance  was  measured  by  questionnaire  and  by 
usag¥^  The  conditions  for  use  were  as  follows.  The  formative  evaluation 
was  designed  such  that  the  operational  maintenance  unit  could  have  the  FSTT 
removed  any  time  they  felt  it  was  not  useful  in  helping  them  accomplish 
their  maintenance  mission.  No  personnel  in  the  unit  were  required  to  use 
the  r$TT.  The  FSTT  was  carried  on  the  unit's  inventory;  therefore,  it  was 
subject  to  inspection  during  the  unit's  Operational  Readiness  Inspection. 

As  such,  it  represented  a  risk  to  the  unit  that  it  would  be  inoperative  at 
the  time  of  inspection  and  thus  reduce  the  unit's  operational  readiness. 
This  was  a  risk  they  would  take  only  if  they  felt  the  benefit  was  greater 
than  the  risk.  Therefore,  usage  for  15  months  was,  in  itself,  an  impor¬ 
tant  measure  of  user  acceptance  by  novices,  experienced  personnel ,  and 
supervisors.  Data  were  recorded  as  to  who  used  it  and  how  much.  These 
data  are  underestimates  of  actual  usage  as  some  records  are  known  to  have 
been  lost. 

The  initial  reaction  of  both  Instructors  and  trainees  to  the  FSTT  was 
very  positive.  Indeed,  until  the  magcard  reader  failure  totally  interrup¬ 
ted  usage,  the  FSTT  was  used  extensively— even  though  there  was  no 
requirement  to  do  so.  The  routine  hardware  failures  that  were  corrected 
promptly  did  not  affect  attitudes  of  on-site  personnel. 

A  single-page,  open-ended  interview  data  collection  form  was 
constructed  to  obtain  data  from  users  on  what  they  learned  and  how  they 
liked  learning  on  the  FSTT.  These  users  included  both  ski 1 1 -level -three 
persons  and  experienced  personnel .  Not  everyone  who  had  used  the  FSTT 
during  the  formative  evaluation  was  available  for  the  questionnaire  inter¬ 
view.  Many  had  been  transferred  or  were  on  TDY  or  leave. 

The  following  comments  were  extracted  from  the  FSTT  Evaluation  Forms 
completed  by  users  and  supervisors  after  experience  with  the  FSTT  during 
the  formative  evaluation.  The  responses  representing  both  motivation/ 
attitude  factors  and  specific  things  learned  are  typical  of  those  obtained: 

Question  1 


How  much  Is  learned  by  novice  maintenance  technicians  by  interacting 
with  the  FSTT? 


o  Logical  troubleshooting  procedures. 
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o  Shows  where  to  start,  step-by-step,  how  to  complete  the 
problem,  but  the  problem  does  not  account  for  preferences 
In  troubleshooting  style. 

o  I  learned  to  break  the  problem  down  Into  smaller  parts  for 
troubleshooting. 

o  Location  of  switches,  where  to  look  to  collect  symptoms. 

Good  maintenance  practices  and  techniques;  they  cut  out 
wasted  time,  learn  efficient  troubleshooting. 

o  The  novice  learns  to  cut  the  problem  up  or  to  narrow  it  down 
to  small  areas. 


o  Step  by  step  troubleshooting. 

o  Gives  you  the  locations  and  names  of  cabinets. 

o  Learn  location  data,  procedures  specific  to  problem  and 

logical  sequence  to  go  by,  required  to  validate  the  writings. 

o  It  teaches  you  exactly  where  to  go  to  troubleshoot. 

o  I  learned  about  cabinet  Identification,  saw  typical  problems. 

o  I  found  it  confusing  and  feel  the  781  write-ups  are  vague. 

o  Feel  some  prior  knowledge  of  the  (flight)  simulator  is  desirable 
before  using  the  FSTT. 

o  Some  are  too  deep  without  classroom. 

Interpretation 

Most  things  users  said  they  learned  are  things  it  was  expected  that 
they  would  learn  "incidental ly"  without  making  enabling  objectives  for 
them  (e.g.,  names  of  cabinets,  locations,  logical  sequence,  etc.).  They 
also  said  they  learned  to  break  a  problem  down  into  smaller  parts  for 
troubleshooting.  We  would  not  expect  users  to  say  they  had  learned  the 
hypothetlco-deductlve  process.  But  we  interpret  their  statements  to  mean 
they  used  the  hypothetico-deductive  process  to  break  the  process  down  into 
steps.  The  FSTT  did  not  provide  step-by-step  procedures  or  logical 
troubleshooting  procedures.  So  if  users  say  that  is  what  they  learned  it 
must  mean  they  learned  the  hypothetico-deductive  process  for  doing  it  by 
themselves.  That  is,  they  achieved  the  end  objective  of  learning  how  to 
solve  problems  (troubleshoot)  on  this  system.  The  last  three  comments 
refer  to  vagueness  and  confusion.  Looking  at  answers  to  other  questions 
by  the  respondents  who  made  these  comments,  the  following  was  noted.  One 
said  simpler  problems  were  not  needed  and  the  FSTT  experience  was  useful 
in  solving  the  problems  on  the  job.  Another  reported  getting  a  lot  out  of 


it,  having  no  complaints,  and  solved  a  problem  on  the  job  like  on  the 
FSTT.  Thus  the  confusing/vague  comments  were  regarded  as  a  "first 
reaction"  and  not  something  that  continued  through  their  experience  on  the 
FSTT.  The  comment  about  "too  deep  without  classroom"  was  made  by  a 
classroom  instructor. 

Question  2 


Describe  the  extent  of  acceptance  by  novice  personnel, 
o  It  was  great;  high  acceptance, 
o  Helpful  for  the  novice. 

o  Fun  to  use;  helped  a  lot  when  I  first  arrived, 
o  First  confused,  then  fun. 
o  Acceptance  is  OK,  here, 
o  Fun  to  use  and  a  challenge. 

o  I  liked  it,  didn't  have  to  worry  about  breaking  the  (flight) 
simulator. 

o  I  like  it;  you  know  you  won't  damage  the  real  equipment, 
o  Very  interesting.  Good, 
o  They  liked  it.  Good. 

Interpretation 

Clearly  this  is  good  acceptance.  Fun  is  mentioned  more  than  once,  as 
is  freedom  from  fear  of  damaging  the  real  equipment.  Challenging  is  men¬ 
tioned.  "Confusing  first  and  then  fun"  could  be  construed  as  "challenging." 

Question  3 


Describe  the  extent  of  acceptance  by  experienced  personnel . 
o  They  ate  it  up. 

o  Definitely  very  helpful,  good  for  cross-training, 
o  Good  when  it  works,  useful;  wish  I  had  more  time  to  use  it. 
o  I  think  I  got  more  out  of  it  than  the  junior  personnel, 
o  Like  a  video  game. 
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Interpretation 


Clearly  accpetance  was  high  by  experienced  personnel.  The  video  game 
comment  is  interpreted  to  mean  that  video  games  don't  instruct  users  as  to 
how  to  get  high  scores.  They  challenge  the  user  to  find  out  how  to  per¬ 
form  better,  just  as  the  FSTT  does. 

Question  4 

What  troubles  in  the  Flight  Simulator  did  you  repair  subsequent  to 
training  on  the  FSTT  in  which  that  training  was  of  help? 

o  Hourglass  on  AR  Scope. 

o  Hourglass  sweep  and  the  off  flag,  motion  on  lamp,  deflection 
in  yoke  servo. 

o  No  video. 

o  I  understood  what  I  was  looking  for. 

o  Harness  release. 

o  Pilot  transmit/receive,  instrument  type. 

o  Attack  Radar  in  cockpit. 

o  Attack  Radar  had  no  video. 

o  Strobing  on  the  Attack  Radar  Monitor. 

Interpretation 

The  problems  mentioned  are  similar  but  not  all  the  same  as  those  on 
FSTT  problems.  Users  are  able  to  solve  similar  problems  not  just  the 
specific  ones  from  FSTT,  thus  meeting  the  objective  of  solving  problems 
not  just  learning  procedures  for  solving  specific  problems. 

Question  5 

About  how  long  should  novice  personnel  use  the  FSTT  as  a  trouble¬ 
shooting  trainer  for  practice  before  using  it  as  a  job  aid? 

o  Answers  ranged  from  a  couple  weeks  to  a  year. 

Interpretation 

The  question  was  not  specific  enough  to  get  a  specific  answer;  and  in 
fact,  the  FSTT  was  not  used  as  a  job  aid.  Also,  users  were  allowed  to  use 
the  FSTT  any  way  they  chose  during  the  formative  evaluation.  Thus,  they 
probably  considered  their  use  and  their  prior  skill  level  in  answering  the 
question. 
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Question  6 


Did  the  FSTT  prepare  you  to  recognize  certain  types  of  failures  when 
they  actually  occurred? 

o  Answers  were  generally  "Yes,"  with  specifics  noted  by  some. 
Interpretation 

Answers  were  consistent  with  those  for  Question  4. 

Question  7 


Would  additional  simpler  problems  be  useful? 

o  Answers  included  “Yes"  and  "No,"  but  more  respondents 
answered  "Yes." 


Interpretation 

Most  users  would  like  a  few  more  simple  problems,  but  an  almost  equal 
number  said  they  would  not.  Making  problems  simpler  could,  of  course, 
reduce  the  challenge. 

Question  8 


Were  the  problems  supplied  challenging? 
o  Answers  were  all  "Yes." 


Interpretation 

The  authors  believe  it  is  better  to  keep  the  problems  challenging 
rather  than  make  them  easier.  The  problems,  as  they  stand,  are  based  on 
malfunctions  that  have  historically  occurred  and  thus  represent  the 
current  level  of  difficulty  on  the  job. 

Question  9 

Were  the  "Aids  to  Problem  Solution"  useful  in  understanding  the  logic 
of  the  problems? 

o  Some  yes,  some  no,  and  several  instances  where  they  were 
not  seen  or  not  available. 


Interpretation 

The  Aids,  reprinted  in  Appendix  A,  represent  a  good  description  of 
what  is  involved  in  each  problem.  They  are  not  an  integral  part  of  the 
learning  process  for  the  FSTT,  as  Indicated  by  the  answers  to  this 
question. 
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User  Performance. 


Analysis  of  FSTT  Printouts 

The  records  of  user  practice  sessions  with  the  FSTT  obtained  from  the 
380  AMS/MAAD  were  examined  to  determine  what  kind  of  learning  experience 
the  users  had.  The  data  from  the  printout  included  the  problem  number, 
the  man-number,  whether  a  correct  repair  was  made,  whether  the  repair  was 
verified,  total  time  per  problem,  number  of  frame  requests,  and  what  frames 
were  requested. 

Records  on  40  individuals  were  returned  from  the  job  site  during  the 
formative  evaluation.  It  is  known  that  individuals  kept  some  of  their 
records  and  some  were  lost.  Judging  from  the  amount  of  paper  used  for  the 
printer  in  comparison  to  the  amount  returned,  about  20  percent  of  the 
records  were  returned  for  analysis.  There  is  no  reason  to  believe  the 
records  returned  were  not  a  representative  sample.  The  40  persons  pro¬ 
duced  167  records,  or  abut  4  records  (trials)  per  person. 

For  each  trial  on  the  FSTT  the  student  entered  an  ID  number.  These 
numbers  were  assigned  for  data  collection  purposes,  rather  than  Social 
Security  Account  Numbers  (SSAs),  because  of  Privacy  Act  restrictions. 

The  data  showed  there  were  14  persons  who  attempted  no  more  than  a 
single  exposure  to  the  FSTT.  Eight  used  ID  numbers  not  assigned  to  the 
roster  for  the  unit.  Perhaps  these  might  have  been  supervisors  who  merely 
wished  to  "try"  the  FSTT.  Such  use  is  of  interest;  therefore,  the  records 
were  included  in  the  general  descriptive  statistics  of  this  section. 

There  were  69  cases,  more  than  one-third  of  the  attempts  to  solve 
problems,  in  which  the  correct  repair  and  verification  were  made  on  the 
first  try.  Successive  attempts  to  solve  a  problem  were  evident  from  the 
fact  that  the  same  ID  number  for  the  same  problems  bore  consecutive  start 
and  stop  times.  Of  interest  was  the  relationship  of  the  number  of  frames 
requested  to  the  amount  of  time  taken  for  a  solution.  The  data  extracted 
from  the  printed  records  were  formatted  in  the  following  way: 

Rec#  Pr#  C.R.  Ver  SCR  ST  FR  1st 
These  abbreviations  have  the  following  meanings: 

Rec#:  The  record  number  which  changes  from  one  sort  to  another. 

(A  "sort"  is  a  sequence  of  items  based  on  either  alpha  or  numeric 

characters.  For  example,  a  sort  can  change  from  alphabetical  to 

numerical .) 

Pr#:  The  problem  number,  from  1  to  30. 

C.R.:  The  yes/no  statement  of  whether  a  correct  repair  was  made. 
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SCR:  The  printed  "combined"  score. 

Ver:  The  yes/no  statement  of  whether  a  verification  run  was  made. 

ST:  System  Time:  minutes  a  person  worked  on  the  problem. 

FR:  The  number  of  frames  accessed  by  the  person. 

1st:  A  number  (1,  2,  or  3)  identifying  which  attempt  this  was 

to  solve  the  same  problem. 

The  data  were  organized  according  to  problem,  and  means  were  deter¬ 
mined  for  SCR,  ST,  and  FR. 

Figure  3  shows  a  plot  of  the  data  for  this  group  of  records.  The 
abscissa  is  calibrated  in  two  scales:  frame  requests  and  system  time  in 
minutes.  The  ordinate  is  calibrated  in  total  score.  As  can  be  seen  from 
this  figure,  when  either  frame  requests  or  system  time  increases,  the 
efficiency  score,  on  the  average,  tends  to  fall.  Linear  regression  calcu¬ 
lations  on  these  data  provide  the  following  equations: 

Score  as  a  function  of  frame  requests,  "FR" 
eq  1 :  S  =  -0.3fr  +  94 

Score  as  a  function  of  system  time,  "ST" 
eq  2:  S  *  -0.2st  +  88 

In  general ,  the  longer  the  person  spent  trying  to  solve  the  problem, 
the  lower  the  efficiency  score.  Also,  the  more  frames  requested,  the 
lower  the  efficiency  score. 

It  was  of  interest  to  remove  the  data  for  those  persons  who  were  known 
to  be  supervisors  or  who  could  not  be  identified  as  other  noncommissioned 
officers  or  civilian  instructors  to  see  if  the  linear  regression  equations 
would  change  significantly.  The  results  were  as  follows: 

Score  as  a  function  of  frame  requests,  reduced  data  base, 
eq  3:  S  =  -0.33fr  +  97.04 

Score  as  a  function  of  system  time,  reduced  data  base, 
eq  4:  S  =  -0.75st  +  92.17 

Comparing  equation  1  with  3,  there  seems  to  be  little  difference 
between  the  two  on  score  as  a  function  of  frame  requests  among  those 
people  who  solved  the  problems  on  their  first  trial  and  also  verified  the 
solutions.  The  group  consisting  of  both  "inexperienced"  and  "experienced" 
seemed  to  have  used  as  many  frames  as  did  the  "inexperienced"  group  alone. 
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Linear  Regression  Equations 
ime  as  Function  of  Frame  Requests 


Comparing  equations  2  and  4,  there  is  a  considerable  difference  in  the 
slope  of  the  equation.  Novice  efficiency  scores  tended  to  fall  off  more 
rapidly  than  did  those  of  the  whole  group,  as  a  function  of  time  on  the 
problem.  This  suggests  that  for  the  same  amount  of  time  on  the  system, 
novices  tended  to  make  more  mistakes,  thereby  getting  a  lower  score  even 
when  they  solved  the  problem.  In  other  words,  experienced  personnel 
tended  to  spend  more  time  considering  their  actions,  whereas  inexperienced 
personnel  tended  to  act.  This  fact  has  implications  for  maintenance  of 
operational  equipment.  Equipment  cannot  be  damaged  when  a  troubleshooter 
is  thinking;  it  can  only  be  damaged  when  the  troubleshooter  is  working  on 
it.  Although  only  some  actions  are  potentially  damaging,  it  is  generally 
a  good  practice  to  think  before  acting. 

There  is  no  learning  style  that  is  particularly  effective  or  ineffec¬ 
tive,  and  a  high  efficiency  score  on  the  first  attempt  is  not  necessarily 
desirable.  A  person  who  examines  the  relationships  of  options  in  order  to 
develop  a  general  problem-solving  skill  may  get  low  scores  on  solving  the 
specific  problem.  The  important  point  is  that  users  continued  to  work  on 
problems  until  they  solved  them--one  way  or  another. 

It  was  evident  from  the  records  of  people  who  tried  a  problem  several 
times  that  the  number  of  frame  requests  was  smaller,  the  efficiency  was 
better,  and  the  time  was  very  short  with  successive  trials.  Typical  per¬ 
formance  patterns  for  those  who  did  not  solve  the  problem  on  the  first 
recorded  attempt  showed  initially  a  long  session  time  (30  to  60  minutes) 
with  a  large  number  of  frame  requests,  and  neither  repair  nor  verification. 
The  second  time  through  the  problem  usually  showed  better  efficiency 
scores,  shorter  session  times  (5  to  10  minutes),  and  fewer  frame  requests 
with  a  correct  repair;  but  often  verification  was  not  accomplished. 
Verifying  the  repair  required  more  frame  requests.  Verification  of  repair 
was  a  source  of  some  consternation  to  FSTT  users.  To  their  perception, 
they  appeared  to  be  deprived  of  a  "solved"  message  when  they  knew  from 
their  efforts  they  had  made  the  proper  repair.  Verification  is  a  process 
that  is  required  on  the  job--it  is  not  an  idiosyncracy  of  the  FSTT. 

The  users  apparently  learned  verification  was  required  on  the  job 
after  they  found  it  was  required  on  the  FSTT.  It  is  obviously  better  to 
learn  this  from  FSTT  experience  than  from  job  experience,  just  as  it  is 
better  to  learn  all  other  aspects  of  job  performance  in  a  situation  where 
no  harm  can  be  done. 

Data  printouts  showed  some  users  were  attempting  several  solutions  in 
a  row  in  order  to  get  a  better  efficiency  score  as  well  as  a  verification 
checkoff.  However,  many  users  would  use  a  second  (or  third)  trial  to 
"explore"  the  less  efficient  paths  to  solution.  This  lowered  their  effi¬ 
ciency  scores  from  their  earlier  scores.  It  generally  took  users  a  half 
hour  or  so  to  solve  the  problem  during  the  first  trial  and  a  few  minutes 
in  later  trial s--except  when  they  were  "exploring." 
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One  interpretation  of  the  results  obtained  from  the  hard  copy  print¬ 
outs  of  performance  on  the  FSTT  is  as  follows.  The  machine's  scores 
derived  from  a  scoring  system  based  on  expert  judgments  of  efficient  and 
rapid  troubleshooting.  Printouts  showed  students  responded  to  this  value 
but  also  ignored  it  as  an  objective  in  order  to  meet  their  own  objective 
of  exploration. 


There  is  an  Inherent  choice  of  exploration  versus  efficiency  in  solving 
problems.  The  scoring  system  rewarded  efficiency.  Users  responded  to 
this  reward  but  also  did  a  significant  amount  of  exploration.  In  effect, 
users  had  three  goals:  to  solve  the  problem,  to  solve  it  efficiently,  and 
to  explore.  Their  behavior  reflects  the  fact  that  they  pursued  the  goals 
alternatively  or  concomitantly  according  to  their  own  inclinations. 

Since  there  is  no  learning  strategy  that  is  known  to  be  better  than  their 
own  inclinations,  it  is  said  here  that  the  behaviors  exhibited  are 
appropriate  to  the  goals. 


The  data  from  the  printer  is  interpreted  to  show  that  the  efficiency 
scoring  system  should  not  be  the  only  basis  used  to  evaluate  what  people 
have  learned  in  this  discovery  learning  situation,  nor  should  the  learning 
situation  be  forced  to  conform  to  the  scoring  system  for  efficiency.  To 
meet  the  exploration  goal,  the  efficiency  score  will  be  lower,  and  this  is 
appropriate.  As  users  develop  their  own  conception  of  what  is  efficient, 
the  efficiency  scoring  system  will  probably  be  ignored  entirely.  At  that 
point  the  FSTT  users  would  join  all  the  other  experts  In  arguing  about 
what  is  most  efficient. 


Courseware  Authoring.  Information  was  provided  to  job  supervisors  on 
how  to  structure  the  fiche,  how  to  affix  and  remove  mattes  to  photographic 
enlargements,  how  to  apply  arrows  and  codes  for  observation,  and  so  forth. 
However,  because  personnel  viewed  the  preparation  of  courseware  as  being 
outside  their  job  descriptions,  the  capability  of  job  site  personnel  to 
develop  their  own  problems  for  the  trainer  could  not  be  tested  in  the  for¬ 
mative  evaluation.  However,  It  seemed  clear  that  they  had  the  capability 
to  do  so  had  they  wished.  This  finding  Indicated  that  a  duty  position 
would  have  to  be  created  or  additional  duties  formally  assigned  to  an 
existing  job,  if  on-site  courseware  development  were  to  become  a  reality. 

Conclusions 

The  Installation  of  the  FSTT  In  the  operational  environment  of  the  380 
AMS/MAAD  showed  that  technicians  would  and  did  use  it  to  solve  trouble¬ 
shooting  problems.  Level  three  personnel  found  It  to  be  a  way  to  gain 
troubleshooting  experience  on  exact  simulations  of  their  equipment.  They 
could  not  get  this  experience  in  the  real  world  even  though  the  equipment 
was  present  on  site.  Their  role  as  helper  did  not  provide  them  practice 
In  troubleshooting  because  troubleshooting  was  done  by  the  level-five  (or 
higher)  technicians.  Level  five  persons  could  familiarize  themselves  with 
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troubles  that  appeared  in  the  "flights”  portion  of  the  simulation  if  they 
were  typically  assigned  to  work  on  "bomb/nav"  problems  and  vice  versa. 
Instructors  on  site  found  that  the  FSTT  could  be  used  to  prepare  personnel 
for  using  data  collection,  troubleshooting  strategies,  and  verifications 
required  on  the  job.  In  addition,  confidence  in  the  trainer  was  amply 
demonstrated  by  the  fact  that  the  maintenance  site  supervisor  permitted 
novice  personnel  to  troubleshoot  operational  equipment  in  the  sunmative 
evaluation. 

It  was  apparent  from  the  formative  evaluation  that,  with  a  reasonable 
amount  of  preventive  maintenance,  the  trainer  was  reliable  and  could  find 
continued  high  acceptance  in  the  380  AMS/MAAf)  and,  therefore,  in  any  other 
similar  organization  for  which  appropriate  problem  sets  were  produced  for 
their  equipment. 


V.  SUMMATIVF.  EVALUATION 


Experimental  Design  and  Test 


Performance  Measurement 


Very  little  job  performance  data  have  been  collected  anywhere  for  any 
purpose  because  job-site  personnel  are  extremely  reluctant  to  have  such 
data  collected.  From  the  earliest  time-and-motion  studies,  both  job 
incumbents  and  supervisors  have  perceived  job  measurements  as  inimical  to 
their  personal  interests.  To  get  around  this,  organizations  conducting 
research  have  used  the  technique  of  taking  over  responsibility  for  job 
equipment  and  creating  a  job-like  situation  where  performance  can  be 
measured.  But  when  a  research  organization  takes  over  the  responsibil ity 
for  the  equipment,  the  job  site  Is  no  longer  an  operational  and  the  data 
are  not  data  from  an  operational  site. 

It  is  not,  therefore,  for  lack  of  measurement  techniques  that  data  are 
almost  never  collected  at  job  sites.  In  the  1950s  more  performance  data 
were  collected  than  in  the  1960s,  and  more  in  the  1960s  than  in  the  1970s 
(Shriver,  Hart,  1975).  Almost  all  performance  data  collected  in  the  past 
30  years  were  collected  under  research  control  of  conditions  in  ways  that 
are  not  possible  on  the  actual  operational  job  site. 

The  present  effort  was  designed  to  maximize  the  opportunities  for 
collecting  actual  job  performance  data.  One  of  the  reasons  for  selecting 
a  job  site  for  experimental  treatment  rather  than  a  school  was  to  increase 
the  probability  that  job  supervisors  would  not  resist  collecting  perform¬ 
ance  data  on  the  job.  It  was  believed  that  if  the  job  site  personnel  saw 
the  trainer/simulator  being  used  by  site  personnel  and  felt  it  was  benefi¬ 
cial,  they  would  allow  performance  data  to  be  collected  at  the  job  site. 
This  generally  proved  to  be  the  case.  In  the  present  effort  the  number  of 
subjects  and  the  number  of  malfunctions  they  were  allowed  to  troubleshoot 
were  not  large,  but  this  is  probably  the  maximum  amount  of  performance 
data  that  could  be  obtained  from  an  operational  job  site. 

The  job  supervisors  did  cooperate  more  in  this  effort  than  in  other 
situations  reported  in  the  literature.  Thus,  this  represents  an  isolated 
case  where  data  on  job  performance  were  collected  on  the  job  site.  The 
only  non-operational  condition  Imposed  by  job-site  supervisors  was  that 
subjects  not  take  any  major  actions  on  the  operational  equipment  before 
saying  what  they  intended  to  do.  As  the  test  event  actually  unfolded,  no 
subjects  were  denied  the  actions  they  wanted  to  take,  so  this  condition 
had  no  practical  effect. 

The  remainder  of  this  section  describes  the  design  and  conduct  of  an 
on-site  job  performance  test  to  determine  if  persons  who  were  trained  in 
troubleshooting  by  means  of  the  FSTT  would  be  able  to  troubleshoot  the 
actual  equipment  better  than  did  those  who  were  not  so  trained. 
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Subject  Population 

In  the  actual  job  situation  there  were  13  persons  who  were  newly 
assigned  and  present  for  duty  during  the  2-month  period  of  job  performance 
tests.  The  research  design  called  for  separating  these  personnel  into  two 
groups  matched  on  relevant  factors. 

These  newly  assigned  personnel  consisted  of  three-level  personnel 
recently  graduated  from  school.  No  five-level  personnel  were  in  the 
target  population  because  they  already  had  about  6  months  of  duty  on 
station.  Because  their  individual  experiences  varied  in  6  months  on  the 
job,  these  personnel  could  not  be  matched  accurately  into  two  groups. 

All  personnel  to  be  tested  had  to  be  on  duty  and  available  for  testing 
during  the  night  shift,  and  they  had  to  have  received  orientation.  The 
orientation  consisted  of  one  week  of  on-site  instruction  designed  to 
familiarize  newcomers  with  the  FB-111A  flight  simulator.  Orientation  is 
conducted  by  experienced  civilian  personnel  and  is  required  by  the  site 
supervisor  before  any  personnel  are  allowed  to  work  on  the  actual  equip¬ 
ment  at  this  site.  The  experimental  group  then  received  FSTT  training; 
the  control  group  did  not  receive  FSTT  training. 

In  the  first  month  of  job  performance  testing,  six  persons  were  iden¬ 
tified  who  were  newly  assigned  to  the  site.  Three  of  them  were  working  on 
the  night  shift  when  testing  began;  the  other  three  started  working  on  the 
night  shift  when  the  normal  shift  rotation  occurred.  There  was  a  good 
match  for  the  personnel  in  these  two  groups  of  three,  so  the  first  three 
were  assigned  to  the  control  group  and  tested  immediately,  while  the 
second  three  were  assigned  to  the  experimental  group,  given  FSTT  training, 
and  tested  when  they  rotated  to  the  night  shift. 

The  second  month  of  job  performance  testing  was  similar  to  the  first. 

An  additional  seven  newly  assigned  personnel  were  available.  They  were 
divided  into  two  matched  groups,  with  the  odd  person  placed  in  the  control 
group.  While  the  testing  was  underway,  one  person  from  the  experimental 
group  became  unavailable.  Another  person,  who  became  available  too  late 
to  complete  FSTT  training  before  being  tested,  was  added  to  the  control 
group.  Thus,  the  total  number  actually  tested  in  the  2  months  was  five  in 
the  experimental  group  and  eight  in  the  control  group. 

A  note  on  the  matching  of  the  personnel  is  In  order.  The  subject 
population  that  arrived  on  the  job  site  in  this  short  period  was  relatively 
homogeneous.  The  recruiting  conditions  that  prevail  over  a  short  period 
like  2  months  brings  Into  the  service  a  relatively  homogeneous  group. 
Recruitment  quotas  were  being  rather  easily  met  at  the  time  these  person¬ 
nel  were  recruited.  They  met  the  aptitude  standards  for  entry  to  the  AFSC 
training  from  which  they  all  graduated.  Thus,  matching  of  Individuals  was 
easily  accomplished. 
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One  person  assigned  to  the  control  group  performed  substantially  less 
well  than  all  the  other  members  of  the  control  group.  This  was  not  fore¬ 
seeable  from  any  background  factors. 


Selecting  the  Problem  Set 

The  same  job  performance  test  was  provided  to  subjects  in  both  groups. 
A  failed  component  or  abnormal  operating  condition  was  inserted  in  the 
actual  equipment  of  the  otherwise  normally  operating  system.  The  malfunc¬ 
tions  selected  for  job  performance  testing  purposes  were  those  which  would 
not  result  in  damage  to  the  operational  flight  simulator.  No  failed  parts 
could  be  inserted  into  the  FB-111  flight  simulator  that  could  conceivably 
disable  the  equipment  or  impose  such  additional  maintenance  demands  as  to 
cause  downtime  on  the  prime  equipment  and  run  the  risk  of  the  equipment 
not  being  available  for  use  by  flight  crews  when  needed.  The  problem  set 
was  thus  limited  to  a  sample  of  6  of  the  30  problems  which  had  been  used 
as  the  training  inventory  on  the  FSTT.  Maintenance  personnel  of  the  380th 
AMS  and  researchers  jointly  selected  the  subset  of  6  problems  for  use  in 
the  evaluation. 

The  problems  were  subsequently  reduced  to  5  because  one  problem  which 
involved  swapping  a  chassis  known  to  be  bad  for  one  known  to  be  good  might 
possibly  cause  additional  damage.  Finally,  during  actual  testing  it  was 
found  that  a  portion  of  the  prime  equipment  was  out  of  order  and  would  not 
support  introduction  of  one  of  these  five  problems;  thus,  the  performance 
test  was  reduced  to  four  malfunction  problems. 


Test  Administration 


To  determine  what  a  reasonable  length  of  time  might  be  for  making  the 
diagnosis  and  repair  for  each  malfunction  problem,  two  five-level  techni¬ 
cians  (supervisors)  solved  the  problems.  It  was  believed  that  five-level 
technicians  would  have  little  difficulty  with  the  malfunctions.  In  fact, 
the  two  technicians  who  participated  in  establishing  the  time  standards 
did  not  differ  from  the  three-level  technicians  in  their  approach  to 
troubleshooting.  After  the  first  few  trials,  the  five-level  technicians 
found  that  their  workload  prevented  further  participation.  What  data  had 
been  collected  were  used  to  set  a  half  hour  time  limit  on  correcting  each 
malfunction  (problem). 

Performance  testing  of  30  minutes  on  each  of  4  problems  would  require 
2  hours  of  testing  per  subject.  Responsible  persons  at  the  380th  AMS 
(MAAD)  agreed  that  a  1  hour  per  person  could  be  spared  from  maintenance 
during  the  night  shift  until  performance  testing  was  completed.  This 
schedule  would  have  resulted  in  2  days  of  testing  per  subject.  In 
actuality,  some  testing  was  allowed  on  the  other  shifts.  This  permitted 
the  testing  to  be  completed  in  less  time. 
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Test  security  procedures  ensured  that  no  one  being  tested  observed  the 
test  malfunction  being  Inserted  into  the  equipment.  It  was  impossible, 
however,  to  keep  the  subjects  from  communicating  about  the  problems  to  one 
another  if  they  chose  to  do  so.  There  was  no  certain  way  to  determine 
whether  this  occurred;  however,  there  were  no  indications  from  their  per¬ 
formance  to  suggest  that  any  subject  had  advance  information. 


Procedure 


A  form  was  devised  for  collecting  data  on  the  troubleshooting  perform¬ 
ance  of  the  subjects  during  their  shift  on  the  job  site.  The  form  was 
used  to  collect  two  kinds  of  data:  first,  success  of  the  subject  in 
solving  the  problem  and  time  to  solutions,  and  second,  a  record  of  every 
step  that  was  actually  taken  In  the  troubleshooting  process.  One  member 
of  the  research  staff  making  observations  was  a  former  FB-111A  maintenance 
technician  who  evaluated  the  actions  taken  by  the  test  subjects  and 
assigned  a  numerical  score  to  each  attempt  at  problem  solution.  The 
numerical  score  was  the  percentage  of  steps  taken  which  were  relevant  to 
the  solution. 

Before  each  problem  administration,  the  malfunction  was  inserted  into 
the  flight  simulator  and  checked  to  ensure  that  the  stated  symptom  pattern 
actually  occurred.  Then,  the  subject  was  called  into  the  bay  and  handed  a 
card  giving  the  statement  of  the  malfunction.  This  is  the  way  a  person  on 
the  shift  normally  receives  a  task.  The  subject  was  told  not  to  take  any 
major  actions  without  telling  the  job  supervisor  what  action  was  planned. 
Otherwise  nothing  was  added  to  the  routine  procedure;  that  Is,  it  was  a 
normal  job  task  so  far  as  the  subject  was  concerned.  No  further  words 
were  spoken.  The  was  started.  The  test  administrator  and  the  shift 
supervisor,  a  five-level  technician,  observed  the  subject;  the  former,  to 
record  what  the  subject  was  doing  and  the  latter,  to  prevent  performance 
of  any  actions  that  could  damage  the  equipment.  The  test  administrator 
recorded  the  cabinets  that  the  subject  went  to  and  each  step  the  subject 
took.  In  all  cases,  the  administrator  noted  the  time  when  either  the 
problem  was  solved  or  the  problem  time  ran  out.  After  completing  the 
problem,  the  subject  was  sent  from  the  bay  to  wait  until  the  next  malfunc¬ 
tion  was  inserted.  Typically,  this  took  about  5  minutes.  Then  the  sub¬ 
ject  was  recalled  to  begin  with  the  next  problem.  Some  of  the  subjects 
solved  the  problems  In  very  short  times,  on  the  order  of  5  minutes,  and 
completed  more  than  the  two  problems  originally  scheduled  for  1-hour 
testing  period. 


Data  Analysis 

Data  collected  on  this  group  of  newly  assigned  technicians  were  ana¬ 
lyzed  using  a  t-test  on  the  means  of  their  times  on  the  problems,  and  also 
on  their  "scores"  as  given  by  the  test  administrator  based  on  observations 
of  their  performance.  Table  1  shows  how  long  each  individual  in  the 
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experimental  group  took  on  each  problem,  whether  solved  or  not.  If  a 
subject  solved  the  problem,  the  time  to  solution  was  the  time  used.  If  a 
subject  used  the  entire  time  (30  minutes)  but  still  did  not  solve  the 
problem,  30  minutes  was  used  as  the  time. 

The  mean  time  for  experimental  subjects  to  correct  the  malfunctions 
was  7  minutes.  The  mean  time  to  complete  each  task  successfully  ranged 
from  about  3  minutes  to  13  minutes.  Among  the  five  subjects  there  was  one 
failure  to  solve  one  problem. 

Table  1. 

Experimental  Group  Time  to  Perform  Problems 


SUBJECT 

PROBLEMS 

(FSTT) 

PWR  3 

VID 

5  HDSET  21 

ILS  23 

El 

3 

12 

3 

13 

E2 

2 

1 

23 

7 

E3 

4 

5 

6 

3 

E4 

7 

1 

30 

14 

E5 

1 

1 

1 

3 

AMOUNT  OF  TIME 

17 

20 

63 

40 

NUMBER  OF  ATTEMPTS 

5 

5 

5 

5 

MEAN  TIME 

3.4 

4 

12.6 

8 

TOTAL  AMOUNT  OF  TIME 
TOTAL  NUMBER  OF  ATTEMPTS 
TOTAL  MEAN  TIME 


Table  2. 


Control 

Group  Time 

to  Perform  Problems 

SUBJECT 

PROBLEMS 

(FSTT) 

PWR  3 

VID  5 

HDSET  21 

ILS  23 

Cl 

5 

4 

13 

3 

C2 

30 

30 

30 

C3 

1 

29 

8 

3 

C4 

5 

10 

8 

C5 

7 

15 

3 

8 

C6 

11 

17 

8 

C7 

1 

1 

30 

23 

C8 

7 

1 

27 

SUM 

86 

101 

136 

135 

NUMBER 

8 

8 

8 

8 

MEAN 

10.75 

12.625 

17 

16.875 

TOTAL  TIME:  458 

COUNT:  32 

MEAN:  14.3125 


In  comparing  the  times.  It  may  be  seen  that  the  experimental  group 
solved  the  problems  in  less  than  half  the  time  required  by  the  control 
group.  The  only  overlap  in  mean  performance  was  that  the  experimental 
group  took  slightly  longer  to  solve  the  most  difficult  problem  than  the 
control  group  took  to  solve  the  easiest  problem.  The  * t *  value,  using  the 
formula: 


_ _ (Ml -M2) _ 

(SUMX12  +  SUMX22)  (N1  +  N2) 

(N1  +  N2  -  2)  (N1  x  N2) 

is  2.496.  This  value  is  significant  at  the  P  <  .05  level  with  12  degrees 
of  freedom  (Llndgren,  McElroth,  1959). 

Tables  3  and  4  show  the  scores  of  experimental  and  control  group  sub¬ 
jects  based  on  the  test  administrator's  evaluation  of  irrelevant/incorrect 
actions  taken  while  each  subject  performed  each  problem.  Ten  points  were 
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Table  4. 


Control  Group  Percentage  Correct  Scores 


SUBJECT 

PROBLEMS 

(FSTT) 

PWR  3 

VI0  5 

DSET  2 

ILS  23 

N1 

100 

70 

0 

100 

N2 

0 

0 

0 

0 

N3 

90 

20 

100 

100 

N4 

100 

60 

100 

0 

N5 

50 

80 

100 

90 

N6 

0 

0 

40 

40 

N7 

100 

100 

60 

40 

N8 

40 

100 

20 

0 

SUM 

480 

430 

420 

370 

NUMBER 

8 

8 

8 

8 

MEAN 

60 

53.75 

52.5 

46.25 

STANDARD  DEVIATION 

43.9 

41.7 

44 

45 

TOTAL  SCORE: 

1700 

COUNT: 

32 

XM2  MEAN: 

53.125 

STANDARD  DEVIATION: 

41.8 

It  might  be  noted  that  significance  is  not  reached  with  such  small 
samples  unless  the  differences  between  the  mean  performances  of  the  two 
groups  are  substantial  and  the  variance  is  small.  In  common  terms  it 
means  that  with  a  small  sample,  there  must  be  little  or  no  overlap  between 
the  two  groups  to  achieve  significance.  Some  researchers  are  reluctant  to 
accept  the  results  of  small  sample  statistics.  Others  feel  that  whenever 
significance  is  reached  with  a  small  N,  it  shows  that  the  effect  of  the 
experimental  variable  is  very  strong.  In  the  present  case  there  is  no 
reason  to  believe  that  differences  in  test  conditions  would  change  the 
results.  Since  there  is  reasonable  consistency  in  the  performances  of  the 
experimental  group,  it  is  unlikely  that  the  results  were  a  function  of 
some  uncontrolled  test  conditions. 

The  scores  prepared  by  the  test  administrator  were  based  on  technical 
judgments  of  the  relevance  of  actions  taken  to  solve  each  problem.  A  per¬ 
son  receiving  a  score  of  100  was  performing  as  efficiently  as  any  person 
could  perform  on  the  task.  Even  experienced  personnel  do  not  perform  with 
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100  percent  efficiency  every  time  they  troubleshoot  a  problem.  These 
scores  were  not  as  objective  as  scores  based  on  the  time  required  to 
troubleshoot  problems,  but  they  were  more  objective  than  qualitative 
statements  that  the  subject  was  "lost"  or  "didn't  know  what  to  do  next." 

In  fact,  supervisors  based  their  judgments  of  personnel  on  technical 
points  such  as  those  used  to  produce  these  scores;  thus,  these  scores 
quantified  the  technical  points  that  supervisors  use  to  evaluate  perform¬ 
ance.  Supervisors  are  interested  in  consistency  of  performance  because 
inconsistent  performance  suggests  that  the  person  cannot  be  trusted  to 
perform  without  injuring  himself  or  damaging  the  equipment. 

It  may  be  noted  that  personnel  who  are  "lost"  or  "stumbling"  are  using 
up  time.  There  is,  therefore,  a  high  correlation  between  performance  time 
and  being  "lost."  In  the  current  results  that  correlation  is  obvious. 

One  more  thing  should  be  noted  in  connection  with  these  scores.  Tech¬ 
nical  observations  provide  a  good  indication  of  whether  or  not  personnel 
passed  information  to  each  other  about  the  problems  they  were  tested  on. 
That  is,  if  a  person  takes  a  particular  action  before  another  action  that 
is  logically  prior  to  it,  it  may  be  an  indication  of  prior  knowledge  about 
the  problem.  The  technical  observer  reported  that  there  were  no  such 
actions  and,  therefore,  no  indication  information  had  been  passed  from 
subject  to  subject. 

The  usefulness  of  these  scores  is  primarily  in  the  insight  they  pro¬ 
vide  to  consistency  of  performance.  In  the  experimental  group,  there  are 
ten  100  percent  scores  out  of  20  scores.  Half  the  performances  were  as 
good  as  they  could  be  on  those  tasks.  Looking  at  the  subjects  shows  that 
two  out  of  five  personnel  were  consistent:  FI  and  F5  with  two  90  scores 
and  the  rest  100s.  Even  very  experienced  personnel  would  not  be  expected 
to  perform  more  consistently.  One  subject  had  scores  of  100,  70,  60,  and 
80.  That  is  good,  but  not  consistent  enough  to  satisfy  some  supervisors. 
The  other  experimental  subject,  F3,  had  scores  too  low  to  trust  him  on  the 
job:  60,  40,  30,  and  100. 

From  looking  at  the  control  group,  it  may  be  seen  that  there  are  ten 
100  scores  out  of  32  scores.  This  shows  that  school  graduates  can  perform 
some  tasks  on  the  job.  On  the  other  hand,  this  confirms  supervisors' 
attitudes  that  newly  assigned  personnel  cannot  be  trusted  to  work  on  their 
own,  for  fear  they  will  injure  themselves  or  damage  the  equipment.  None 
of  the  control  group  subjects  was  consistent.  Two  of  the  control  subjects 
who  had  two  100  scores  also  have  a  zero  score  and  another  relatively  poor 
score  on  other  tasks  (N1-N4).  One  of  the  other  subjects  with  two  100 
scores  has  relatively  poor  scores  on  the  other  two  tasks  (N7).  One  of  the 
other  100  scores  was  received  by  a  subject  who  was  a  predominantly  poor 
performer  on  the  other  tasks  (N8).  The  worst  performance  by  any  subject 
in  the  control  group  is  all  zeros  (N2).  The  next  worst  is  two  zeros  and 
two  40s  (N6).  The  problem  with  newly  assigned  personnel,  as  shown  by  the 
control  group  performance,  is  lack  of  consistency  as  much  as  lack  of  job 
competence.  Consistency  is  what  job  supervisors  look  for  to  build  their 
trust  enough  to  allow  newly  assigned  personnel  to  work  on  the  job. 


It  was  noted  that  one  control  group  subject  failed  all  problems  (all 
zeros).  In  order  to  show  that  the  overall  results  were  not  a  function  of 
this  one  aberrant  case,  the  subject's  score  was  raised  from  zero  to  the 
mean  for  the  control  group.  The  statistics  indicated  the  same  level  of 
confidence  in  the  significance  of  the  difference  between  the  control  and 
experimental  groups  when  this  subject's  scores  were  changed  from  zero  to 
the  average  control  group  scores  on  each  task. 


Discussion 

The  experimental  group  clearly  differed  from  the  control  group  on  all 
measures  of  job  performance  on  the  job  tasks  used  in  this  experiment.  The 
difference  Is  statistically  significant. 

There  was  room  for  Improvement  In  the  consistency  of  one  or  two 
experimental  group  subjects;  however,  none  of  the  control  group  subjects 
was  consistent.  Again,  job  supervisors  are  interested  in  consistency. 

They  avoid  giving  responsibll ity  to  newly  assigned  personnel.  The  fact 
that  there  is  no  consistency  in  the  control  group  subjects'  performance 
confirms  the  job  supervisor's  traditional  viewpoint  that  newly  assigned 
personnel  cannot  be  trusted  to  work  on  their  own  on  the  job.  With  most  of 
the  work  force  being  first-term  personnel,  waiting  for  them  to  develop 
consistency,  only  to  have  them  transferred  or  leave  the  service,  is  not 
cost-effective.  Clearly  there  is  a  need  to  increase  consistency  of  job 
performance  early  so  that  first-termers  can  perform  useful  work,  rather 
than  serve  only  in  a  helper  role. 

The  demonstrated  capability  of  the  FSTT  to  improve  the  performance  of 
newly  assigned  personnel  and  make  more  than  half  of  them  consistent  per¬ 
formers  is  an  outcome  which  Is  greatly  needed  in  Air  Force  maintenance. 

It  should  be  noted  that  this  outcome  was  obtained  in  a  relatively  casual 
manner.  There  was  no  command  emphasis  on  the  FSTT  experience.  The  FSTT 
was  merely  a  device  on  trial ,  yet  it  demonstrated  a  substantial  effect. 
This  result  was  with  a  trivial  Investment  in  time  and  material  on  the  part 
of  the  squadron,  yet  at  least  half  the  personnel  receiving  FSTT  experience 
became  consistent  performers  whereas  none  of  those  without  FSTT  experience 
did  so.  It  is  highly  probable  that  some  command  emphasis  and  another  40 
or  50  hours  of  FSTT  experience  would  have  an  even  greater  effect--such  as 
making  all  newly  arrived  personnel  consistent  performers.  This  represents 
the  next  step  for  future  research.  The  FSTT  has  demonstrated  what  it  can 
do  on  its  own.  The  next  step  Is  to  make  it  the  basis  for  a  program  to  see 
if  job  supervisors  can  change  their  attitude  toward  new  personnel  and  give 
them  job  responsibilities  wherein  they  can  produce  useful  work  rather  than 
perform  as  helpers  for  6  months  to  a  year. 


Conclusions 


1.  A  nominal  amount  of  training  experience  on  the  FSTT  significantly 
improves  the  job  performance  capability  of  personnel  newly  assigned 
to  maintenance  jobs  (on  the  FB-111  Flight  Simulator). 

2.  At  least  half  the  personnel  with  (nominal)  FSTT  training  experience 
demonstrated  consistency  in  job  performance,  whereas  none  of  the 
newly  assigned  personnel  without  FSTT  experience  demonstrated 
consistency. 

3.  It  is  highly  likely  that  a  courseware  program  like  that  used  on 
the  FSTT  hardware,  with  the  objective  of  giving  newly  assigned 
personnel  work  to  do  on  their  own  (rather  than  serve  as  helpers 
for  6  months),  would  demonstrate  greater  work  productivity  in 
maintenance  units. 

4.  The  FSTT  developed  for  this  project  provided  the  random  access 
capability  required  to  make  the  courseware  effective.  There  are 
now  cheaper  and  more  reliable  hardware  devices  that  provide  quick 
access  to  any  frame  of  visual  image  in  storage  according  to  a 
software  program. 


Recommendations 


1.  Use  the  FSTT  courseware  as  the  basis  for  a  program  to  obtain 
useful  work  from  newly  assigned  maintenance  personnel  and,  as 
a  result,  increase  unit  productivity. 

2.  Make  videotape  images  of  the  FSTT  courseware  frames  and  produce 
a  videodisk  to  replace  the  microfiche  component  of  the  FSTT 
hardware. 

3.  Direct  the  program  toward  flight  simulator  maintenance  and  expand 
it  to  operational  aircraft  maintenance  if  it  proves  effective  for 
flight  simulator  maintenance. 
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FSTT 


AIDS  TO  PROBLEM  SOLUTION 


PROBLEM  #1 

IISAF  FORM  781  STATEMENT:  "HOURGLASS  SWEEP  ON  COCKPIT  AR  INDICATOR" 

GENERAL:  The  Attack  Radar  Indicator  in  the  simulator  cockpit  has  been 

modified  so  that  it  contains  somewhat  different  circuitry  from 
the  AR  indicator  in  the  aircraft.  But  it  still  has  many  simi¬ 
larities.  The  AR  monitor  on  the  console,  on  the  other  hand,  is 
quite  different.  A  significant  difference  lays  in  how  many 
sweeps  are  generated  and  displayed.  The  cockpit  indicator  has 
two  circuits  relating  to  sweep  that  (1)  produce  a  sectored  PPI 
shaped  display,  and  (2)  run  the  sweep  out  in  range.  The  Yoke 
Servo  circuit  is  concerned  with  azimuth,  while  the  sweep 
circuit,  or  gamma  servo,  is  responsible  for  range.  Troubles  in 
either  of  these  circuits  will  show  up  locally.  That  is,  the 
symptom  will  show  up  in  the  indicator  but  not  on  the  console. 
The  hourglass  sweep  is  called  that  because  of  its  resemblance 
to  an  hourglass. 

SPECIFIC:  Start  with  a  strategy  page  to  see  what  options  for  trouble¬ 
shooting  are  available. 

Check  switch  settings  to  determine  that  normal  operating  set¬ 
tings  are  in  effect,  and  that  a  false  trouble  is  not  present. 

Check  if  the  symptom  is  present  and  localized  or  generalized. 

If  local  in  nature,  isolate  to  the  removable  part. 


EMPLOYEE  NUMBER :  _ 

DATE  PROBLEM  WAS  TRIED: 
SCORE : 
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AIDS  TO  PROBLEM  SOLUTION 


PROBLEM  #2 

USAF  FORM  781  STATEMENT:  “WINDSHIELD  WIPER  SYMPTOM  ON  AR  COCKPIT 

INDICATOR" 

GENERAL:  The  Attack  Radar  Indicator  in  the  simulator  cockpit  differs 

from  the  monitor  on  the  console.  Generally,  the  console  moni¬ 
tor  repeats  the  display  that  appears  on  the  cockpit  display. 
When  the  pilot  changes  ranges,  the  monitor  will  show  the 
changed  ranges.  But  if  the  CRT  in  the  indicator  in  the  cockpit 
burned  out,  the  console  monitor  would  continue  to  display  video 
since  culture  returns  are  generated  by  the  Landmass  system  and 
are  shared  by  both  displays.  In  other  words,  there  are  some 
kinds  of  troubles  that  are  specific  to  the  hardware  in  each 
physical  location.  When  such  troubles  occur,  the  effects  are 
not  necessarily  seen  on  both  displays. 

The  Gamma  servo  circuitry  and  the  Yoke  servo  circuitry  in  the 
AR  cockpit  indicator  will  affect  the  shape  of  the  display  seen 
on  that  indicator.  Troubles  in  either  of  these  circuits  will 
not  necessarily  be  displayed  on  the  console  monitor. 

The  Yoke  servo  circuitry  affects  the  shape  of  the  "fan."  The 
Gamma  servo  circuitry  affects  range  sweep. 

SPECIFIC:  Start  with  a  strategy  page  to  see  what  options  are  available. 

Check  switch  settings  to  ensure  normal  operation. 

Check  if  symptom  is  present,  and  where  located. 


EMPLOYEE  NUMBER: 


DATE  PROBLEM  WAS  TRIED: 


SCORE: 
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AIDS  TO  PR OB L KM  SOLUTION 


PROBLEM  #3 

IISAF  FORM  781  STATEMENT:  "POWER  LOST  ( 28VDC)" 

GENERAL:  The  simulator  uses  many  different  power  levels  of  both  AC  and 

DC.  In  troubleshooting  power  failures,  it  is  useful  to  divide 
the  systems  into  AC  systems  and  DC  systems,  as  they  both  have 
their  own  methods  of  control.  Simulator  power  is  input  to 
cabinet  4,  MAIN  SIMULATOR  POWER.  Cabinet  5,  consisting  of 
power  supplies  and  circuit  breakers,  permits  selective 
distribution  of  power  to  simulator  systems  and  subsystems.  If 
observation  of  power  panels  and  of  circuit  breaker  panels  fails 
to  localize  the  problem,  go  to  the  power  distribution  prints 
and  try  to  trace  the  circuits. 

SPECIFIC:  Check  the  781.  What  power  was  lost?  What  controls  the  power 
supplied  to  the  element(s)  or  systems  that  lost  power?  If  it 
is  not  a  power  supply  that  failed,  it  could  be  a  circuit 
breaker.  Power  supplies  are  usually  located  near  circuit 
breaker  panels,  or  incorporate  circuit  breakers  (though  not 
always) . 


EMPLOYEE  NUMBER:  _ 

DATE  PRORLEM  WAS  TRIED: 
SCORE: 
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AIDS  TO  PROBLEM  SOLUTION 


PROBLEM  #4 

USAF  FORM  781  STATEMENT:  "NO  VIDEO  DISPLAYED" 

GENERAL:  Many  write-ups  are  either  misleading  or  ambiguous  in  practice. 

The  reason  for  this  is  that  the  person  doing  the  writing  may 
not  be  able  to  describe  the  symptom  meaningfully.  Technicians 
must  learn  to  live  with  this  fact,  and  treat  the  write-up  as  an 
attempt  to  provide  information,  but  not  to  put  too  much  faith 
in  it.  For  example,  "NO  VIDEO  DISPLAYED"  may  mean  totally 
blank  screens  to  one  person,  missing  culture  returns  to  another, 
and  so  on.  Generally,  it  Is  wise  to  examine  all  displays  that 
would  show  video,  both  on  the  console  and  in  the  cockpit. 
Determine  what  kinds  of  Images  are  in  fact  displayed,  and  on 
which  scope.  Since  range  marks  are  not  generated  in  the 
Landmass,  presence  or  absence  of  these  marks  helps  to  isolate 
the  trouble.  Even  if  the  FSS  and  film  plate  are  functioning 
properly.  If  the  antenna  Is  not  sweeping,  the  culture  returns 
will  not  be  displayed. 

SPECIFIC:  Check  the  strategy  page  of  troubleshooting  strategy  options. 

Verify  the  presence  of  the  reported  symptom,  and  try  to 
determine  what  the  781  means. 

Check  status  of  controls  and  configuration  of  the  AR. 

Verify  presence  of  antenna  motion. 

Check  AR  video  distribution. 


EMPLOYEE  NUMBER: 


DATE  PROBLEM  WAS  TRIED: 


SCORE: 


AIDS  TO  PROBLEM  SOLUTION 


PROBLEM  #5 

USAF  FORM  781  STATEMENT:  "INCORRECT  AR  VIDEO" 

GENERAL:  There  are  many  ways  in  which  the  AR  video  can  be  incorrect. 

The  write-up  is  not  specific  as  to  where  the  symptom  was 
observed.  Since  the  console  AR  monitor  and  the  AR  indicator 
in  the  cockpit  can  have  different  displays  If  one  or  the  other 
Is  faulty,  it  is  important  to  check  both  places.  Also,  it  is 
important  to  see  if  the  mode  of  operation  is  consistent  with 
the  display  observed.  Check  switch  position. 

SPECIFIC:  Check  the  strategy  frames  to  see  what  troubleshooting  paths  are 
available  to  you. 

Check  both  displays. 

Check  switch  settings. 

Verify  the  symptom. 


EMPLOYEE  NUMBER: 


DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #6 

USAF  FORM  781  STATEMENT:  "NO  VIDEO  ON  ANY  SCOPE  -  NO  TIME  SIGNAL" 

GENERAL:  There  are  three  scopes  to  consider:  the  ARS  (attack  radar  set, 

cockpit),  the  ARM  (attack  radar  monitor,  console),  and  the 
TFR/TAR  (terrain  following  radar-terrain  avoidance  radar,  both 
cockpit  and  console).  In  this  case  there  is  nothing  on  any  of 
the  scopes.  This  symptom  requires  starting  a  system  checkout 
at  a  broad  and  high  level.  Since  all  scopes  are  affected,  the 
failure  is  common  to  all  three  systems.  The  fault  is  something 
that  either  controls  the  video  to  the  scopes,  or  has  to  do  with 
its  production  and  delivery  to  the  scopes.  A  clue  is  provided 
in  the  781:  "no  time  signal."  While  this  is  general  and  does 
not  pinpoint  a  system,  it  does  afford  some  help  because  the 
simulation  is  computer  controlled  and  gating,  controlling  RLMS/ 
Tactics  operations,  whichever  mode  is  selected,  is  critical. 

A  second  clue:  Computer/ linkage  timing  is  good.  (Do  not 
consider  the  computer  suspect.) 

SPECIFIC:  Verify  loss  of  video.  Find  a  system  (or  systems)  common  to  all 
scopes. 

Check  for  operation  or  loss  of  operation  from  a  high  level, 
narrowing  down  to  a  system  (or  systems).  What  was  meant  by 
saying  "loss  of  time  signal"? 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 
SCORE : 
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PROBLEM  #7 

USAF  FORM  781  STATEMENT:  "CROSS-HAIRS  DO  NOT  FALL  ON  TARGET" 

GENERAL:  As  the  FB-111  flies  over  areas  of  the  earth,  its  geographic 

coordinates  depend  on  its  speed  and  direction  of  flight.  In 
the  simulator,  the  transparency  (filmplate  or  "map")  in  the 
optics  cabinet  of  the  RLMS  duplicates  the  topographic  features 
found  in  the  real  world  so  that  realistic  navigation  and  radar 
operations  can  take  place.  The  coordinates  are  updated  in  a 
way  that  will  correspond  with  the  motion  of  the  aircraft  rela¬ 
tive  to  the  ground.  This  requires  precise  alignments  between 
the  RLMS  optics  and  the  aircraft  display  systems  (navigation 
and  radar/tactics  display  instrumentation).  The  simulator 
incorporates  the  means  by  which  non-linearities  can  be 
corrected  (ground  map-transparency  alignments)  so  that  the  crew 
can  be  presented  accurate  data.  If,  for  instance,  one  were  to 
attempt  navigating  to  PAFB,  the  known  latitude,  longitude,  and 
elevation  could  be  entered  to  the  the  Navigation  Display  Unit, 
and  if  within  radar  range,  one  could  position  cursors  on  the 
ARS  over  PAFB;  there  would  be  no  disagreement,  the  cross-hairs 
would  be  over  the  base,  and  the  NDU  would  show  PAFB's 
geographic  coordinates. 

SPECIFIC:  Check  the  ARS  and  NDU  and  try  to  get  the  cross-hairs  to  target 
the  latitude-longitude  data  in  the  NDU. 

Go  through  RLMS  and  check  optics. 
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PROBLEM  #8 

USAF  FORM  781  STATEMENT:  "CANNOT  COMPLETE  FIDUCIAL  ALIGNMENT 

(OSCILLATIONS)" 

GENERAL:  A  fiducial  alignment  is  a  procedure  whereby  the  optics  cabinet 

filmplate  is  positioned  to  allow  the  FSS  to  scan  each  fiducial 
mark  found  on  the  four  sides  of  the  filmplate.  When  aligned, 
the  filmplate  data  will  accurately  reflect  the  latitude  and 
longitude  position  of  the  aircraft/ground  returns  etc.).  The 
filmplate  is  positioned  by  servos.  The  filmplate  motion  com¬ 
puter  is  the  subsystem  that  controls  filmplate  positioning. 

The  write-up  indicates  there  are  "oscillations"  occurring,  and 
therefore  it  is  Impossible  to  keep  the  scan  centered  over  the 
desired  spot. 

SPECIFIC:  The  RLMS  uses  a  small  general -purpose  computer,  the  Raytheon 
703.  This  computer  is  tied  in  to  allow  the  RLMS  to  perform, 
among  other  things,  computations  involving  latitude/longitude 
data.  These  coordinates  are  used  to  position  the  filmplate 
transparency  according  to  aircraft  flight  attitude/conditions. 
It  is  not  necessary  to  perform  the  "fiducial  alignment"  in  this 
problem.  The  objective  is  to  remove  the  oscillations  from  the 
system  so  that  a  fiducial  alignment  can  be  done. 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #9 

1JSAF  FORM  781  STATEMENT:  "MIRROR  IMAGE  ON  AR  VIDEO" 

GENERAL:  The  simulator  uses  many  systems  to  simulate  radars.  These 

include:  optics  (initial  ground  return  data),  DTG,  VGE  (air 
targets,  video  pulses,  triggers  etc.),  servos  (antenna  motion 
simulation),  and  the  necessary  computers  and  handshaking 
systems  (linkage,  DTG,  BCM.703  Raytheon,  etc.).  A  failure  in 
any  one  of  these  systems  results  in  a  faulty  radar  system 
simulation.  This  ranges  from  degraded  performance  to  total 
failure.  The  ARS  in  this  problem  is  operational,  but  a  "mirror 
image"  has  appeared. 

SPECIFIC:  Since  both  indicators  show  the  condition,  the  problem  must  be 
in  circuits  common  to  both.  Is  there  anything  in  the  radar 
computers  that  could  cause  this  condition?  The  TFR  is  OK.  (Is 
gross  video  data  from  optics,  etc.  good  then?) 

Check  the  ARS-ARM  scopes. 

Check  sweep  circuits. 
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PROBLEM  #10 

IJSAF  FORM  781  STATEMENT:  "OFF-FLAG  ON  MACH  INDICATOR" 

GENERAL:  The  Mach  airspeed  indicator  is  part  of  the  cockpit  flight 

instrumentation.  The  indicator  on  the  console  is  a  digital 
readout  unit.  The  one  in  the  cockpit  is  an  actual  aircraft 
indicator.  Cockpit  indicators  are.  for  the  most  part,  synchro- 
driven  devices.  Data  driving  them  come  from  the  ESRD  system  in 
cabinet  14.  These  indicators,  however,  also  receive  logic  data 
and  power  from  simulated  flight  director  computer  systems 
(simulated  by  the  simulator  main  computer).  The  MAS  (Mach 
airspeed  indicator)  uses  a  "black  box,"  located  beneath  the 
pilot's  area  of  the  cockpit.  The  ESRD  system  inputs  to  this 
box,  which  in  turn  outputs  to  the  indicator. 

SPECIFIC:  Determine  if  the  write-up  is  valid.  In  some  cases  of  fault 

determination,  it  is  necessary  to  track  more  than  one  possible 
fault:  keep  this  in  mind. 
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DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #11 

IJSAF  FORM  781  STATEMENT:  "HSI  BEARING  POINTER  DOES  NOT  ROTATE" 

GENERAL:  The  HSI  (Horizontal  Situation  Indicator)  in  the  cockpit  is  used 

in  flight  navigation.  It  contains  a  compass  card,  a  "to-from" 
indicator,  a  distance  readout,  bearing  pointer,  a  course 
deviation  bar,  course  set  window  and  a  power  on-off  flag.  The 
HSI  unit  is  not  simulated;  it  is  an  actual  aircraft  instrument. 
However,  drive  voltages  and  logic  status  supplying  the  HSI  are 
simulated.  Cockpit  flight  instrumentation  uses  synchros  driven 
by  26-volt,  400  Hz  power. 

SPECIFIC:  Verify  the  781. 

Find  the  cause  of  the  write-up  by  checking  HSI  circuitry 
(simulator  circuits,  of  course,  not  those  internal  to  the  HSI). 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #12 

USAF  FORM  781  STATEMENT:  "MOTION  'ON*  LIGHTS  (PLATFORM)  REMAIN  LIT 

AFTER  MOTION  HAS  BEEN  DESELECTED" 

GENERAL:  On  the  left  and  right  side  stairways  providing  cockpit  platform 

access  are  two  warning  lights.  "Flight  in  Progress"  and 
"Motion  On."  One  of  these  two  warning  lights  remains  lit. 

SPECIFIC:  How  Is  the  warning  light  deactivated?  Has  the  system  been 
turned  off  or  is  it  still  on? 

Check  the  TOs. 
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PROBLEM  #13 

USAF  FORM  781  STATEMENT:  "NUMBER  1  UHF  DOES  NOT  TRANSMIT" 

GENERAL:  The  FB-111  uses  various  devices/equipment  for  communications 

as  well  as  navigation  aids.  There  are  two  UHF  systems  onboard 
the  FB-111:  numbers  1  and  2.  The  simulated  transmission  faci¬ 
lities  in  the  cockpit  are  exact  replicas  of  actual  can-sets 
used  in  the  aircraft.  Those  on  the  console,  however,  are 
comprised  of  the  tactics  and  flight  Instructor's  transmit  and 
monitor  circuits.  The  crew  members  in  the  cockpit  transmit  to 
the  console.  By  observing  console  displays,  console  instructors 
can  ensure  that  the  range,  frequencies,  and  ID  data  are  such 
that  communication  could  actually  take  place  in  the  real  world. 
Console  personnel  can  then  acknowledge  reception  of  crew  member 
messages. 

SPECIFIC:  The  UHF  system  uses  a  system  of  selectable  frequencies.  Those 
on  the  console  should  match  those  In  the  cockpit.  How  is 
transmission  effected  (cockpit  to  console)? 

Verify  the  write-up. 

Check  to  see  if  all  transmission  Is  out,  or  if  it  Is  specific 
to  one  circuit. 

Check  the  interface  circuit  prints  and  trace  signals. 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #14 

USAF  FORM  781  STATEMENT:  "INSTRUCTOR  CONSOLE  LIGHT  DOES  NOT  LIGHT  WHEN 

SWITCHLIGHT  IS  DEPRESSED" 

GENERAL:  Console  swltchllghts  are  switches  with  a  lightbulb  in  them. 

When  they  are  lit,  they  signify  that  the  system/subsystem  is 
active  (or  should  be).  In  other  words,  by  depressing  a  switch 
light  (which  then  illuminates),  whatever  that  switchlight 
controls  should  become  active  (or  inactive). 

SPECIFIC:  Is  the  lightbulb  good?  Which  switchlight  is  the  781  referring 
to?  Refer  to  the  prints  and  check  the  lights/voltage  input(s) 


EMPLOYEE  NUMBER: 


DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #15 

USAF  FORM  781  STATEMENT:  “INSTRUCTOR  CONSOLE  DOES  NOT  SHOW  AR  RANGE 

SELECTED  IN  COCKPIT" 

GENERAL:  When  a  crew  member,  or  technician,  alters  a  switch  position  in 

the  cockpit,  the  event  is  observable  at  the  console.  It  may  be 
as  simple  as  a  single  light  coming  on  or  going  off,  or  it  could 
be  a  series  of  lights  coming  on  or  going  off.  It  could  be  an 
involved  intermix  of  status  display  changes,  lights  coming  on 
or  going  off,  and  extensive  reorientations  of  flight  and  tactics 
displays.  The  Attack  Radar  in  the  cockpit  can  be  operated  in  a 
variety  of  modes  and  ranges  and  the  actions  of  the  individual 
manipulating  and  ARS  controls  can  be  observed  at  the  Attack 
Radar  Monitor  panel ,  tactics  console. 

SPECIFIC:  Verify  the  write-up. 

Check  the  cockpit  to  ensure  the  range  switch  is  configured 
properly  to  allow  the  console  light  to  be  checked. 

Check  the  TO  prints  for  the  circuit  path. 

Verify  whatever  repair  actions  taken  result  in  proper  simulator 
operation. 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #16 

USAF  FORM  781  STATEMENT:  "MOTION  SYSTEM  DOES  NOT  ERECT  PROPERLY" 

GENERAL:  The  motion  system  is  an  electro-mechanical -hydraul ic  system 

that  the  simulator  uses  to  move  the  cockpit  in  response  to  pilot 
control  stick  movement,  and  external  conditions,  like  bumpiness 
in  nap-of-the-earth  flying.  There  are  five  hydraulic  cylinders 
that  control  cockpit  attitude.  One  of  the  three  vertical 
cylinders  could  be  responsible  for  incorrect  motion  system 
erection.  Or,  something  that  is  common  to  all  three  systems 
could  be  at  fault. 

The  first  step  would  be  to  find  out  what  troubleshooting 
strategies  are  available.  The  principal  strategy  page  is 
located  at  trainer  code  55. 

The  "mule"  or  hydraulic  pimping  unit  must  be  operational  if  the 
motion  system  is  to  erect  properly.  PRESSURES  required  must  be 
available. 

SPECIFIC:  Check  the  options  or  measurements  available  under  "hydraulic 
strategies." 

Under  normal  conditions,  the  pressure  gauges  at  the  mule  do  not 
show  any  pressure,  but  it  is  possible  to  momentarily  turn  these 
valves  on  to  measure  the  pressure  in  the  system. 

If  there  Is  a  leak,  hydraulic  fluid  will  be  noticed. 

If  the  pressure  Is  too  low,  proper  erection  cannot  be  obtained, 
and  repressurization  will  be  required.  Diagrams  found  at  02 
and  03  will  lead  to  measurements  of  the  pressure  and  appropriate 
corrective  action.  Once  proper  pressure  has  been  achieved,  the 
gauges  will  read  3000  psl  when  the  platform  is  fully  erected. 

In  practice,  this  Is  a  transitory  thing  taking  perhaps  up  to  10 
seconds  to  erect  fully,  during  which  time  the  gauge  will  show  a 
lesser  pressure. 
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PROBLEM  #17 

USAF  FORM  781  STATEMENT:  "LAGGING  RESPONSE  TO  CONTROL  STICK" 

GENERAL:  The  control  sticks  (or  control  columns,  as  they  are  referred  to 

in  the  TOs)  are  mechanically  connected  via  a  system  of  bell- 
cranks,  arms  and  levers  to  hydraulic  servos  and  transducers. 
These  allow  the  simulator  to  follow-up  stick  movements  with  the 
right  "feel"  feedback  from  the  control  column  as  well  as  cockpit 
platform  motion.  Unless  there  is  an  obvious  clue  to  possible 
fault,  whether  it  be  hydraulic,  mechanical,  or  electrical.  It 
is  a  good  idea  to  check  mechanical  elements  briefly.  Then, 
check  for  electrical  inputs. 

SPECIFIC:  The  term  "control  loading"  applies  to  the  flight  control  system 
used  in  the  simulator  whereby  flight  control  Inputs  initiated 
within  the  cockpit  are  sensed  and  output  via  the  mechanical  - 
hydraulic-electrical  system  to  the  linkage  computer.  Do  not 
attempt  performing  a  control  loading  alignment  unless  sure  It 
is  needed.  Try  to  check  the  obvious,  or  if  not  obvious,  then 
easily  accessible  Items  first.  The  simulator  is  provided  with 
built-in  test  equipment;  try  to  use  the  "dlgibox"  for  core 
checks. 
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PROBLEM  #18 

USAF  FORM  781  STATEMENT:  "MAIN  DC  LOAD  CANNOT  BE  APPLIED" 

GENERAL:  The  simulator  uses  a  wide  variety  of  power  supplies.  In 

addition  to  the  power  supplies,  circuit  breakers  are  used  to 
control  power  routing  and  prevent  damage  due  to  voltage  surges 
and  overloads.  The  power  supplies  use  internal  controls  as 
well  as  feedback  to  provide  constant  and  linear  outputs.  Power 
distribution  throughout  the  simulator  fans  out  from  cabinets  4 
and  5.  From  the  circuit  breaker  panels  on  cabinet  5,  power  to 
user  systems  can  be  selectively  removed  or  applied  as  desired. 

SPECIFIC:  Check  the  781  write-up.  Where  is  power  removed  from? 

By  checking  the  power  distribution  prints  found  in  the  TOs,  it 
is  possible  to  trace  the  circuit  paths  of  the  various  power 
systems  from  cabinet  4  to  the  final  using  element. 

Check  power  supply  panel  Indicators  for  normal  or  abnormal 
values. 

If  a  circuit  breaker  trips  immediately  upon  reset,  there  is  a 
chance  it  Is  defective;  also,  the  input  to  it  is  abnormally 
high. 


EMPLOYEE  NUMBER: 


DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #19 

USAF  FORM  781  STATEMENT:  "NO  TANKER  BEACON  SYMBOL  ON  AR” 

GENERAL:  In  refueling  operations,  the  Attack  Radar  Set  is  used  as  a 

means  of  locating  the  tanker  and  then  tracking  (homing)  to  the 
tanker  for  the  refuel.  The  ARS  has  to  be  put  into  the  air  mode 
and  the  target  (tanker)  acquired  via  the  tracking  handle  for  a 
lock-on.  (The  aircraft  can  fly  to  the  tanker  automatically.) 
The  tanker  should  appear  on  the  ARS  scope  as  a  series  of  lines 
(strobes)  patterned  according  to  the  code  selected. 

SPECIFIC:  Determine  if  the  ARS  is  configured  for  air  mode.  Determine 

what  procedures  provide  the  necessary  video  on  the  ARS  and  ARM 
scopes.  (Since  this  is  a  tactical  function,  the  DTG  and  VGE 
should  concern  you.) 

Verify  whatever  repair  actions  you  have  taken  by  observing  the 
ARS  and  ARM  scopes  to  see  that  a  tanker  beacon  becomes  visible. 


EMPLOYEE  NUMBER :  _ 

DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #20 

USAF  FORM  781  STATEMENT:  “ANTENNA  TILT  NOT  WITHIN  LIMITS" 

GENERAL:  The  actual  antenna  within  the  nosecone  of  the  FB-111  is 

engineered  to  operate  within  certain  tilt  angle  limitations. 

The  simulator  Is  designed  to  the  same  limitations  on  the 
simulated  radar.  Instead  of  using  servos  to  position  an 
antenna  dish,  the  servos  in  the  simulator  are  driven  to  the 
same  angles/limits  as  in  the  actual  aircraft  and  shaft  encoders 
digitize  the  information  to  be  processed  in  other  circuits. 

SPECIFIC:  It  is  suggested  that  the  areas  you  choose  to  check  out  be 

chosen  with  the  TO  material  referenced  with  it;  i.e.,  if  you 
are  going  to  check  the  outputs  of  the  radar  receiver  computer, 
go  to  the  -1  (TO). 

Check  the  AR  interface  system  prints. 

Verify  the  limits  corresponding  to  the  required  values. 


EMPLOYEE  NUMBER: 


DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #21 

USAF  FORM  781  STATEMENT:  "PILOTS  CAN'T  TRANSMIT  OR  RECEIVE" 

GENERAL:  Intercommunications  circuits  and  systems  allow  the  pilot  and 

WSSO  to  communicate  to  each  other.  There  are  various  circuit 
paths  to  consider,  but  if  there  is  no  communication  at  all  (hot 
mike  function  allows  the  lines  to  stay  open),  then  It  could  be 
necessary  to  check  the  systems/power  supplies  In  common  with 
one  another. 

SPECIFIC:  Check  items  that  receive  a  lot  of  use  and  are  subjected  to 
mechanical  stress. 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 
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PROBLEM  #22 

USAF  FORM  781  STATEMENT:  "NO  AZIMUTH  MARKER  ON  EITHER  RADAR  SET" 

GENERAL:  The  radar  scope  azimuth  marker  allows  a  radar  set  operator/ 

observer  to  relate  radar  returns  to  a  given  aircraft  heading. 
The  azimuth  marker  is  moved  to  run  through  the  return.  The 
angle  between  the  marker  on  aircraft  heading  is  used  in  target 
acquisition.  The  azimuth  marker  on  the  console  Is  slaved  to 
the  one  in  the  cockpit.  (Since  both  ARS  and  ARM  receive  the 
same  signal  from  interface,  the  displays  should  be  the  same.) 
There  is  an  azimuth  marker  video  intensity  control  pot  on  the 
ARS  panel  beneath  the  ARS  CRT,  however,  this  pot  has  no  effect 
in  this  problem. 

SPECIFIC:  The  azimuth  marker  pulse  is  a  video  pulse,  produced  by  the 

video  circuitry  in  cabinet  6,  which  receives  commands  and  data 
from  DTG,  cabinet  19. 

Check  the  ARS  and  ARM  (scopes). 

Check  video  output  from  cabinet  6. 
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PROBLEM  #23 

USAF  FORM  781  STATEMENT:  "ILS  IDENTIFIER  INOPERATIVE" 

GENERAL:  The  Instrument  Landing  System  (ILS)  allows  for  blind  let-downs 

using  cockpit  instrumentation  that  is  receiving  radio  data  from 
ground  sources.  The  pilot  flies  down  to  the  selected  field  by 
means  of  pitch  and  course  deviations  bars  on  the  ADI  as  well  as 
audio  tone  cues  that  are  specific  to  the  airfield  being 
approached. 

SPECIFIC:  Verify  the  781.  Check  to  see  if  the  Identifier  is  present. 

Check  the  ILS  control  panel  in  the  cockpit  for  its  set-up.  The 
set-up  of  the  ILS  panel  must  be  right  for  the  field  desired. 

The  ILS,  when  working  properly,  presents  deviations  on  the  ADI, 
aural  cues  (identifier),  as  well  as  outer  and  middle  marker 
cues. 
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PROBLEM  #24 

USAF  FORM  781  STATEMENT:  "CONSOLE  MICROPHONES  ARE  WEAK  AND  GARBLED" 

GENERAL:  The  console  microphones  are  connected  into  whatever  communica¬ 

tion  system  is  selected,  and  permits  console-to-cockpit, 
consol e-to-console  (tactics  to  flights),  and  hot  mike  intercom, 
at  a  volume  level  adjustable  from  the  console.  Cabinet  15 
contains  circuits,  amplifiers,  filters  and  oscillators  used  in 
simulating  the  intercom/communications  systems.  (Cabinet  15 
contains  circuits,  amplifiers,  filters,  and  oscillators  used  in 
simulating  the  Intercom/communications  systems.  (Cabinet  15 
does  not  have  them  all,  however.  Most  of  them  are  to  be  found 
in  the  audio  and  aural  cue  cabinet.) 

SPECIFIC:  Verify  the  781. 

Check  the  TOs  to  determine  the  circuit  paths  the  various 
communications  systems  take. 

Verify  the  repair  by  checking  the  system  out. 
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PROBLEM  #25 

USAF  FORM  781  STATEMENT:  "TFR  CLIMB/DIVE  BAR  INOPERATIVE" 

GENERAL:  The  TFR  (Terrain  Following  Radar)  system  allows  for  blind  let¬ 

downs,  and  fast,  low-level,  terrain  following  flight.  There 
are  basically  two  ways  of  using  it  in  flight.  The  first  way  is 
by  selecting  the  auto  TF  function,  presetting  flight  controls 
and  allowing  the  TFR  unit  to  "fly"  the  aircraft  handsoff.  The 
other  way  is  to  fly  the  aircraft  manually,  but  to  use  the  pitch 
steering  bar,  visible  on  the  ADI,  which  becomes  available  if 
the  manual  mode  is  selected.  There  are  several  things  to  keep 
in  mind  concerning  the  TFR  systan.  It  is  adjustable  to  fly  the 
aircraft  at  several  preset  levels.  If  the  TFR  is  not  in  auto 
TF  and  manual  TF  flight  is  selected,  the  pilot  has  to  rely  on 
aural  and  visual  cues.  If  over  flat  terrain,  the  ADI  pitch  bar 
should  be  somewhere  near  the  center  of  the  ADI  when  the  aircraft 
attitude  is  level  flight  and  the  ground  clearance  is  set  within 
the  flight  envelope. 

SPECIFIC:  Check  the  781  write-up.  Observe  the  TFR  control  panel;  observe 
the  ADI. 

Check  the  prints. 

Verify  whatever  repair  action  taken  restores  the  use  of  the  ADI 
pitch  bar. 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 
SCORE: 
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FSTT 


AIDS  TO  PROBLEM  SOLUTION 


PROBLEM  #26 

USAF  FORM  781  STATEMENT:  “NO  LOW  LEVEL  REFLECTANCE" 

GENERAL:  The  Attack  Radar  is  capable  of  several  modes  of  operation.  The 

returns  displayed  on  the  AR  monitor  and  cockpit  Indicator  can 
include  such  topographic  characteristics  as  returns  from 
electric  power  transmission  lines  and  poles.  In  the  simulator, 
the  circuitry  that  processes  such  returns  is  called  the  Low 
Level  Reflectance  circuits.  Without  returns  of  this  type,  the 
display  may  appear  to  be  acceptable.  In  general  ,  these  returns 
may  be  difficult  to  see  on  the  scopes  even  when  present  because 
of  the  presence  of  other  returns  that  may  be  stronger,  and  mask 
them. 

One  generally  useful  strategy  to  adopt  when  verifying  this 
symptom  is  to  set  the  simulator  to  the  location  where  it  is 
known  that  low-level  reflectance  would  be  available  If  the 
circuitry  was  operational. 

This  strategy  may  or  may  not  be  available  to  you. 

SPECIFIC:  Check  the  strategy  page  to  see  what  troubleshooting  options  are 
available  to  you. 

Observe  the  AR  scopes,  both  for  signals  and  settings. 

Find  the  RLMS  circuitry  dealing  with  low  level  reflectance,  and 
check  for  proper  signals. 


EMPLOYEE  NUMBER: 


DATE  PROBLEM  WAS  TRIED: 


SCORE: 
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FSTT 


AIDS  TO  PROBLEM  SOLUTION 


PROBLEM  #27 

USAF  FORM  781  STATEMENT:  "NAVIGATOR  CANNOT  COMMUNICATE  WITH  PILOT" 

GENERAL:  There  are  several  switches  in  the  cockpit  that  pilot  and 

navigator  each  can  operate  in  order  to  activate  different  modes 
of  communication.  There  is  a  switch  on  the  throttles,  used  in 
air  to  ground  communication,  and  there  are  other  switches  on 
the  floorboards  for  use  in  the  intercom  system.  In  addition, 
the  setting  of  switches  on  the  UHF  and  HF  radios  have  a  bearing 
on  the  success  of  voice  communication.  The  simulator  does  not 
actually  use  a  radio,  but  does  have  switch  circuitry  that 
mimics  the  circuits  In  the  aircraft. 

SPECIFIC:  Check  the  strategy  page  to  see  what  troubleshooting  options  are 
available  to  you. 

It  may  be  necessary  to  make  voltage  measurements  within  the 
Intercom  circuitry. 


EMPLOYEE  NUMBER :  _ 

PATE  PROBLEM  WAS  TRIED: 
SCORE: 
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FSTT 


AIDS  TO  PRORLEM  SOLUTION 


PROBLEM  #28 

USAF  FORM  781  STATEMENT:  "NO  LOW  LEVEL  REFLECTANCE" 

GENERAL:  The  video  displays  on  the  AR  scopes  simulate  cultural  returns 

that  are  interpretable  as  reflectance  data  from  specific 
objects  on  the  ground.  The  filmplate  is  color  coded  to  provide 
data  for  every  portion  of  a  ground  return  visible  on  the  real 
radar.  Thse  levels  of  intensity /illumination  are  called  steps, 
and  each  one  is  used  to  correspond  to  a  given  ground  effect/ 
intensity.  The  WSSO,  or  technician,  observes  the  ARS  scope 
hoping  to  see  proper  returns  for  the  given  area  of  radar  scan. 
By  noting  a  lack  of  "low-level  reflectance,"  you  can  refer 
immediately  to  the  RLMS  subsystem  that  has  to  do  with  this 
problem. 

SPECIFIC:  Check  the  AR  scopes.  Go  to  the  RLMS  section  and  check  to  see 
how  the  filmplate  video  is  being  processed  (or  isn't  being 
processed). 


EMPLOYEE  NllMRER:  _ 

DATE  PROBLEM  WAS  TRIED: 
SCORE: 
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FSTT 


AIDS  TO  PROBLEM  SOLUTION 


PROBLEM  #29 

USAF  FORM  781  STATEMENT:  "RIGHT  SHOULDER  STRAP  LOCKS" 

GENERAL:  The  shoulder  strap  or  Inertial  reel  assembly  is  part  of  the 

crew's  equipment  for  restraining  them  during  rapid  aircraft 
accelerations,  and  during  emergency  ejection.  Controls  for 
straps  are  located  In  the  cockpit. 

SPECIFIC:  Check  the  strategy  frame  to  determine  an  effective  trouble¬ 
shooting  strategy. 

Ensure  that  the  reported  symptom  actually  exists. 

Check  for  broken  cables,  or  misadjustments. 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 
SCORE: 
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AIDS  TO  PROBLEM  SOLUTION 


PROBLEM  #30 

USAF  FORM  781  STATEMENT:  "ENGINE  PERFORMANCE  INDICATORS  DO  NOT 

CORRESPOND  WITH  THROTTLE  POSITIONS" 

GENERAL:  When  the  throttles  In  the  cockpit  are  moved,  there  are  many 

cues  that  show  the  resulting  effect  of  throttle  repositioning. 

If  they  are  advanced,  fuel  flow  increases,  the  RPM  indicators 
(tachometers)  spool  upwards,  and  other  engine  performance  indi¬ 
cators  show  the  resulting  effects  of  increased  engine  thrust. 

For  the  computer  to  know  engine  status,  inputs  are  necessary 
so  that  the  computer  can  acknowledge  the  action  taken  in  the 
cockpit. 

SPECIFIC:  Verify  the  write-up. 

Check  the  throttles.  There  Is  a  way  the  motion  of  the  throttles 
is  being  translated  into  electrical  Inputs  to  the  computer. 

Find  the  location,  and  check  the  cores.  Check  the  TOs. 


EMPLOYEE  NUMBER:  _ 

DATE  PROBLEM  WAS  TRIED: 


SCORE: 
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-  AUTHORING  TECHNIQUES  FOR  DEVELOPING  FSTT  SIMULATION  PROBLEMS 


APPENDIX  B  -  AUTHORING  TECHNIQUES  FOR 
DEVELOPING  FSTT  SIMULATION  PROBLEMS 


The  content  of  courseware  problems  for  the  FSTT  Is  produced  by 
courseware  authoring  activities.  This  is  a  formal  description  of  the 
authoring  activities.  In  practice,  the  process  is  simpler  than  It  sounds 
in  a  verbal  specification.  And,  in  practice,  shortcuts  in  documenting  the 
procedures  may  be  taken.  The  following  description  presents  specifics  for 
the  process  of  courseware  development. 


PROCEDURES 

The  first  step  in  authoring  a  problem  for  the  FSTT  is  to  identify  and 
select  a  fault  upon  y/bich  a  troubleshooting  problem  can  be  based.  The  next 
step  is  to  identify  a  set  of  troubleshooting  steps  that  will  efficiently 
isolate  the  fault.  While  each  author  will  have  a  preferred  method  for 
isolating  the  fault,  each  method  should  be  effective  in  achieving  the 
solution  and  the  repair  of  the  fault.  After  the  specific  sets  of  observa¬ 
tions  and  required  tests  have  been  determined,  alternative  sets  or  paths  to 
solution  should  also  be  identified.  Alternative  troubleshooting  paths  are 
those  which  are  not  as  efficient  as  the  optimal  path.  An  alternative  path 
can  be  absolutely  wrong,  in  the  sense  that  taking  an  action  on  this  path  is 
dangerous  to  personnel  or  equipment,  but  to  be  effective,  alternative  paths 
should  be  reasonable  and  appear  sensible  to  the  user.  The  author  may  obtain 
information  about  the  troubleshooting  path  from  a  functional  analysis  of 
the  system  and  from  the  technical  orders  for  the  system.  An  author  who  is 
an  experienced  troubleshooter  of  the  operational  system  already  possesses 
sufficient  knowledge  of  the  system  but  still  must  explicitly  identify  all 
the  steps  and  data. 

The  next  step  is  to  specify  the  observations  and  specific  test  points 
at  which  to  make  measurements.  These  data  should  be  entered  in  the  field 
data  collection  book  as  the  guides  for  collecting  data  in  the  field.  Data 
already  in  the  resource  file  may  satisfy  certain  requirements;  however, 
until  that  is  known,  it  is  best  to  consider  that  all  data  will  be  obtained 
through  on-site  observations. 

Next,  the  author  should  consult  the  resource  file.  The  resource 
(described  elsewhere)  contains  the  selected  photographs  that  were  used  in 
the  construction  of  the  previous  problems.  It  Is,  obviously,  possible 
that  some  of  the  photographs  needed  In  the  new  problem  can  be  found  in  the 
resource  file.  If  so,  there  is  no  requirement  to  take  new  photographs 
unless  there  have  been  equipment  changes. 

If  additional  photographs  are  needed,  with  the  field  data  collection 
workbook  in  hand  and  the  camera  equipment  available,  the  author  will  go  to 
the  field  and  arrange  to  take  the  pictures.  These  photographs  will  show 
equipment  locations.  At  some  locations,  symptoms  of  the  fault  will  be 
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displayed.  At  others,  test  points  will  be  available  so  that  test  equipment 
can  be  installed  and  pictures  taken  of  the  test  Instrument  readings.  The 
equipment  must  be  set  up  for  the  operating  conditions  under  which  the 
trouble  occurs  including  Insertion  of  the  fault  selected.  Any  test  equip¬ 
ment  needed  should  also  be  taken  to  the  field  to  provide  needed  pictures. 
The  field  data  collection  workbook  can  be  used  for  recording  data  about 
the  photographs  being  taken  and  what  they  show. 

When  photographs  taken  In  the  field  are  finished  and  are  judged 
suitable  for  developing  a  problem,  the  author  will  identify  those  requiring 
enlargements.  Otherwise,  if  transparencies  are  to  be  used  directly,  they 
will  be  part  of  the  problem  content. 


Developing  the  Problem 

When  all  the  necessary  materials  are  available,  development  of  the 
problem  should  begin  through  use  of  flche  assignment  sheets  and  the  fiche 
planning  board. 

All  FSTT  problems  begin  with  a  statement  of  the  problem,  as  it  appears 
on  a  Form  781,  on  the  left-hand  viewer.  On  the  right  viewer,  a  diagram 
which  shows  the  overall  physical  layout  of  equipment  In  the  simulator  bay 
is  displayed.  These  frames  are  accessed  with  the  00  code  to  start  each 
problem. 

For  each  potential  action  available  to  the  user,  a  corresponding  code 
must  be  assigned.  Codes  are  assigned  to  locations  on  the  top  level  diagram 
and  will  allow  access  to  detailed  pictures  of  the  system  components  rele¬ 
vant  to  the  problem;  e.g.,  the  cabinet  that  a  troubleshooter  would  go  to 
collect  Information.  The  same  procedure  should  be  followed  for  each 
location  which  gives  a  choice  on  the  top  level  diagram.  This  copy  may  be 
photographs,  cropped  pictures,  cropped  parts  of  the  enlargements,  or  parts 
of  photostats  of  the  TOs.  They  must  be  identified  and  placed  on  the  fiche 
assignment  sheets,  and  a  miniature  copy  placed  on  the  fiche  planning  board. 

When  the  codes  have  been  assigned  to  the  copy,  they  should  be  checked 
off  against  the  fiche  assignment  sheets.  Fiche  assignment  sheets  are  a 
convenient  control  on  which  fiche  assignments  have  been  made,  which  codes 
have  been  allocated,  which  codes  call  other  codes,  and  what  the  content  is 
for  a  particular  code  location.  After  the  codes  have  been  assigned,  the 
content  must  then  be  assigned  to  the  fiche  locations  corresponding  to 
those  codes.  The  author  should  mark  up  the  fiche  assignment  sheet  and 
place  the  Images  of  that  content  on  the  fiche  planning  board.  There 
should  be  some  representation  (either  a  miniature,  or  a  note  Identifying 
the  material)  of  the  Image  to  be  assigned  to  a  fiche  location  placed  In 
each  of  the  cells  used  In  the  fiche  planning  board. 

Next,  the  author  should  assign  the  test  point  data  codes.  The  photo¬ 
graphs  taken  of  test  points  In  the  field  are  a  usual  source  of  such 
information.  It  Is  possible  to  represent  the  normal  condition  by  means  of 
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the  word  "normal"  rather  than  the  actual  value,  but  it  is  preferable  to 
use  a  value.  If  the  test  point  data  is  a  voltage  and  there  is  no  interest 
in  having  students  practice  reading  voltage  scales  in  the  problem,  then 
the  voltage  obtained  at  a  test  point  may  be  printed  on  the  copy  (e.g.,  128 
VDC) . 

The  optimal  patch  (strategy)  through  the  problem  should  be  plotted 
first.  The  author  simply  assigns  the  specific  diagrams  and  figures  to 
places  within  the  problem  on  the  fiche  planning  board  such  that  it  will  be 
possible  always  to  keep  previous  frames  in  place  while  accessing  data  on 
the  other  viewer.  This  is  a  preferred  solution  although  it  is  not  abso¬ 
lutely  mandatory.  When  all  of  the  required  copy  for  the  optimal  path, 
including  test  point  data,  have  been  assigned  to  the  fiche  locations,  the 
author  then  goes  back  to  the  strategy  page  and  assigns  the  alternate  paths. 
These  may  include  sane  of  the  copy  previously  indicated  on  the  optimal 
path. 


Arranging  the  Abort  Paths 

In  some  problems,  it  may  be  desirable  to  include  an  action  that  would 
cause  some  danger.  If  such  an  action  is  provided  as  an  option  on  a  path, 
it  should  be  specially  coded  by  assigning  an  ADC  to  that  particular  frame. 
This  code  will  cause  the  problem  to  "abort"  if  it  is  selected.  After  an 
abort,  the  user  must  start  the  problem  over  again. 


Selecting  Assistance  Frames 

The  codes  07  through  18  are  the  frame  codes  at  which  it  is  possible  to 
obtain  assistance.  The  author  may  use  theory  of  operation,  relevant 
suggestions  for  action,  or  other  assistance  on  these  frames.  Generally 
anything  that  reviews  the  situation  the  user  is  in  provides  help.  The 
help  should  not  just  tell  the  user  what  to  do  next  to  get  out  of  the 
situation. 


Checking  the  Code  Assignments 

When  all  of  the  copy  has  been  assigned,  the  author  should  then  ensure 
that  each  frame  selected  brings  up  copy  at  the  locations  called  for.  That 
is,  if  the  top  level  diagram  calls  for  a  picture  at  code  70,  then  it  would 
be  important  to  find  that  there  has  been  content  assigned  to  70  and  that 
that  content  is  what  is  supposed  to  be  there.  This  is  a  verification 
process  that  ensures  all  fiche  codes  have  been  appropriately  designated. 
The  fiche  assignment  sheets  should  also  contain  the  listings  of  the  "from" 
and  "to"  calls. 

When  the  author  is  satisfied  that  this  Is  the  case,  the  codes  should 
be  placed  on  the  copy.  The  coding  activities  essentially  fall  into  three 
main  areas.  The  first  of  these  is  the  coding  of  the  color  enlargements. 
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The  color  enlargements  may  be  masked  so  as  to  exclude  certain  areas 
which  are  of  no  Interest  to  the  problem  at  hand.  They  may  be  masked  In 
either  the  horizontal  or  vertical  format.  Photographs  should  be  masked, 
and  then  tissue  paper  or  vellum  placed  over  the  masks.  The  courseware 
author  then  Indicates  by  pencil  on  the  tissue  paper  where  specific  codes 
should  be  placed  on  the  copy.  These  codes  have  previously  been  Identified 
on  the  facsimile  of  the  photograph  used  In  the  fiche  planning  board,  so  It 
Is  a  matter  of  transcribing  those  data,  as  well  as  the  frame  location  code, 
onto  the  cover  sheets  for  the  enlargements.  (For  example,  21A23.  21 
refers  to  Problem  21,  A  refers  to  fiche  A,  and  23  refers  to  the  23rd  frame 
on  the  fiche.) 


After  the  author  has  designated  the  trainer  codes,  the  copy  can  be 
coded  using  specially  printed  sticky-back  tabs.  White  and  black  arrows 
are  used  to  connect  each  code  to  the  specific  hardware  item  to  which  the 
trainer  code  refers.  The  white-  and  black -bordered  arrows  are  used 
because  they  are  visible  under  every  circumstance  of  dark  photograph  or 
light  photograph.  When  all  the  color  enlargements  have  been  so  coded  by 
the  author,  they  are  then  turned  over  to  the  copy  preparer  (who  may  also 
be  the  author)  and  Included  In  the  enlargement.  Subsequently,  they  are 
photographed  by  means  of  the  copy  stand  In  order  to  obtain  color 
transparencies.  (See  Micropublishing  for  description  of  that  process.) 


Preparing  Photostat 


The  second  preparation  activity  Is  In  connection  with  the  stats. 

These  are  photostatic  copies  of  pages  that  have  been  selected  from  the 
technical  orders  (TOs).  The  photostatic  material  itself  Is  reasonably 
tough  material  that  will  withstand  the  attachment  and  removal  of  sticky- 
back  code  tabs.  In  assigning  codes  to  the  photostats,  the  author  uses  a 
xerox  copy  of  the  page  and  marks  up  that  copy  for  subsequent  use  as  a 
guide  to  coding  the  stat.  Again,  frame  location  codes  are  placed  In  the 
lower  left-hand  corner,  and  the  copy  is  mounted  on  sheets  of  paper 
measuring  14  x  11  Inches.  This  size  of  paper  provides  a  constant  size  of 
Image  for  the  fiche  maker  and  ensures  that  the  full  image  frame  is 
occupied  even  though  the  content  of  the  frame  may  occupy  less  space. 


Test  Point  Photographs 


The  waveforms  used  In  connection  with  observations  of  an  oscilloscope 
are  placed  on  the  black/white  fiche.  Here  the  negatives  taken  from  the 
P/N-type  polarold  oscilloscope  camera  film  can  serve  as  the  basic  source 
for  the  waveform.  Each  negative  Is  subsequently  xeroxed  and  the  copy 
placed  on  8-l/2-1nch  by  11-Inch  paper.  The  paper  subsequently  Is  coded 
with  the  frame  location  code  so  there  Is  no  ambiguity  about  v4»ere  to  place 
the  image  within  the  frame.  There  are  no  access  codes  placed  on  the  copy 
of  test  point  readings. 
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When  the  copy  has  been  prepared  for  a  fiche,  a  flche  layout  sheet  is 
prepared.  A  fiche  layout  sheet  is  a  symbolic  representation  of  the 
corresponding  fiche  planning  board.  All  those  locations  on  the  fiche 
which  are  not  used  are  so  indicated  by  an  X. 


Fiche  Layout 

The  fiche  layout  sheets  are  the  principal  means  of  communication 
between  the  courseware  author  and  the  fiche  maker.  Essentially,  they  tell 
the  fiche  maker  which  fiche  location  frames  have  been  identified.  In 
addition,  of  course,  each  frame  of  copy  submitted  is  identified  by  a  loca¬ 
tion  number.  The  fiche  layout  sheet  is  also  used  as  an  Inventory  control 
document  to  ensure  that  every  frame  to  be  microficbed  has  been  prepared 
and  is  ready  to  be  sent  to  the  micropublisher. 


Header 


The  header  is  a  title  identifying  the  fiche.  It  will  provide  the 
problem  number  and  any  other  codes  that  may  be  desired;  for  example,  the 
revision  number  and  the  data  for  publication  of  that  problem.  The 
material  to  be  microfiched  should  then  be  assembled  Into  a  package  and 
sent  to  the  micropublisher  with  header  and  instructions  for  publication. 


Transparencies  for  Color  Fiche 

Transparencies  for  use  in  color  micrographics  should  be  retained  in 
the  paper  holders  vrtiich  are  supplied  by  the  film  processor  doing  the  work. 
These  paper  holders  should  be  labeled  with  the  frame  identification  of  the 
image.  During  the  photographing  of  the  enlargements,  the  photographer  may 
have  used  several  different  aperture  settings  In  order  to  achieve  the 
proper  exposure.  The  author  should  select  the  most  appropriate  photograph 
and  indicate  on  the  covers  which  one  was  selected.  This  can  be  done  by 
using  a  check  mark  or  an  X,  or  some  other  symbol  of  that  sort. 

In  construction  of  the  color  microfiche,  not  every  frame  contains  a 
transparency  or  color  image.  In  those  cases  where  line  art  or  text  are 
used,  the  copy  should  be  placed  on  blue  paper  (e.g.,  Hammermill  Four  Bond 
Blue  #10164-2).  The  blue  paper  provides  a  means  of  making  copies  of  the 
content  and  at  the  same  time  gives  the  microfiche  maker  less  trouble  in 
obtaining  color  balance  for  the  color  microfiche.  The  set  of  materials  to 
be  sent  to  the  color  microfiche  maker,  therefore,  consists  of  a  set  of 
labeled  color  transparencies  and  the  set  of  blue  paper  hard  copy,  plus  the 
fiche  layout  sheets  and  the  transmittal  letter.  Before  any  of  the 
materials  have  been  sent  to  the  microfiche  maker,  archive  copies  of  the 
materials  are  prepared.  The  archive  copies  consist  of  xeroxed  (or  other 
dry  copier)  reproductions  of  the  materials.  These  archive  copies  are 
important  because,  in  the  event  a  revision  needs  to  be  made,  the  specific 
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coding  for  a  given  stat  or  enlargement  can  be  reconstructed  more  easily. 
Hence,  archive  copies  are  prepared  and  stored  with  the  materials  that  are 
in  the  resource  file  after  they  leave  the  lesson  development  file. 

After  the  fiche  are  returned  from  the  micropublisher,  they  should  be 
loaded  into  the  carriers  and  brought  into  the  FSTT  for  examination.  The 
points  to  look  for  are  discussed  in  the  section  on  micropublishing. 
Following  examination  of  the  fiche  for  copy  quality,  if  satisfactory,  then 
the  magnetic  card  coding  can  be  undertaken. 


Magnetic  Card  Coding  -  Introduction 

In  using  the  FSTT,  users  receive  feedback  on  their  selection  of  frames 
in  the  form  of  messages  that  appear  on  the  in-line  display  above  the  left 
microfiche  viewer.  The  specific  words  used  as  the  qualitative  feedback 
repertoire  are:  correct,  incorrect,  relevant,  irrelevant,  and  the  +  and  - 
symbols.  When  the  action  taken  by  the  user  is  judged  to  be  very  ineffi¬ 
cient,  three  minus  signs  are  used.  Where  a  choice  on  the  optimal  path  is 
entered,  three  plus  signs  appear. 

In  addition,  other  messages  are  provided  under  certain  circumstances. 
For  example,  if  the  user  chooses  to  make  a  repair  that  would  result  in 
damage  either  to  the  equipment  or  self,  the  message  "EQUIPMENT  DAMAGED- 
-ABORT"  could  be  given.  (A  listing  of  the  messages  is  provided  on  the  Mag 
Card  Coding  Guide  at  the  end  of  this  Appendix.)  The  FSTT  has  been  hard¬ 
wired  for  this  set  of  messages,  but  it  is  the  author  who  must  elect  whether 
to  use  the  messages. 

In  addition  to  the  messages  that  appear  as  part  of  the  normal  solution 
to  a  problem,  other  messages  that  have  been  incorporated  into  the  FSTT 
allow  the  author  to  interact  with  the  device  during  the  authoring  phase. 

An  example  of  a  message  that  can  appear  is  "BAD  DATA  READ  ON  MAG  CARD." 

This  message  occurs  if  an  attempt  to  load  a  blank  mag  card  was  made. 

Each  addressable  fiche  frame  associated  with  a  given  problem  is 
accessed  by  inputting  the  two  digit  hex  code  associated  with  that  image. 
These  access  codes  have  been  previously  referred  to  as  "trainer"  codes  or 
sometimes  just  "codes."  When  a  code  is  entered  by  means  of  the  trainee 
keypad,  the  associated  fiche  frame  is  displayed  on  the  appropriate  fiche 
viewer.  When  a  code  is  entered  on  the  trainee  keypad,  the  electronic 
controller  examines  it  and  determines  from  the  stored  values  associated 
with  that  code  and  the  conditions  existing  at  the  time,  what  efficiency 
score  to  display  as  a  result.  The  in-line  display  then  shows  the  results 
of  that  calculation.  However,  in  order  for  the  FSTT  to  perform  this 
calculation,  the  values  associated  with  the  trainer  codes  must  be  recorded 
on  the  magnetic  card  for  the  lesson. 

The  solution  of  a  given  troubleshooting  problem  can  be  performed  by 
acquiring  the  needed  data  from  available  sources  in  a  logical  fashion. 
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The  images  selected  for  use  in  any  given  lesson  were  selected  by  the 
courseware  author  because  they  constitute  a  feasible  set  of  images  to 
represent  places  at  which  to  obtain  information  about  the  condition  of  the 
equipment  and,  thereby,  to  solve  the  problem.  The  fiche  would  contain 
photographs  of  the  places  at  which  a  troubleshooter  would  examine  and 
collect  additional  data  such  as  images  of  waveforms  as  seen  on  oscillo¬ 
scopes  or  control  settings  on  equipment.  The  fiche  also  contain  pages 
from  the  TOs  that  would  be  relevant  to  the  decisions  to  be  made  as  to 
where  to  make  tests.  The  fiche  also  contain  photographs  of  panels  or 
cabinets  which  do  not  display  data;  the  user  must  observe  a  more  detailed 
view  in  order  to  obtain  the  data  needed. 

If  the  problem  contained  only  that  subset  of  data  or  images  which 
were  relevant  to  the  optimal  solution  of  a  given  problem,  it  would  not 
adequately  represent  the  real  world,  in  which  both  relevant  and  irrelevant 
pages  of  a  TO  are  available.  Moreover,  since  in  the  real  world  a  trainee 
could  examine  not  only  the  relevant  test  points  but  also  the  test  points 
that  have  correct  values,  the  FSTT  has  been  designed  to  provide  for  the 
examining  of  test  points  and  collecting  data,  even  though  those  test 
points  might  not  be  the  most  efficient  ones  for  the  solution  of  a  given 
problem.  Consequently,  every  frame  that  is  accessible  must  be  evaluated 
by  the  author  as  to  its  importance  to  the  efficient  (optimal)  solution  of 
the  problem.  In  those  cases  where  the  abort  messages  are  to  be  used,  it 
is  clear  that  the  abort  should  result  in  a  "no  solution"  of  the  problem. 

A  table  showing  the  hex  weighting  codes  and  the  corresponding  messages 
is  provided  at  the  end  of  this  section.  Hex  weighting  codes  from  1  through 
F  are  used.  The  hex  code  0  is  the  default  value  and  simply  means  that  if  0 
were  inserted  in  a  given  mag  card  location,  the  entry  would  not  result  in 
score  computation  nor  would  there  be  any  feedback  provided  for  accessing 
that  particular  frame.  Seven  values  are  used  In  computing  the  score:  0 
through  6.  The  maximum  value  (6)  is  given  in  the  case  of  a  highly  relevant 
(highly  efficient)  or  correct  choice.  On  the  other  hand,  the  value  0  (as 
selected  by  hex  code  1  or  9)  may  be  used  for  the  highly  irrelevant  (highly 
inefficient)  case,  or  highly  incorrect  case. 

The  FSTT  has  been  designed  with  the  objective  of  ensuring  that  if  the 
optimal  path  is  taken  through  a  problem,  a  score  of  100  percent  would  be 
possible.  The  only  way  that  such  a  score  can  occur  is  if  every  frame 
associated  with  the  most  efficient  path  is  scored  either  as  a  hex  code  "7" 
(meaning  relevant  three  pluses)  or  "F"  (meaning  correct  three  pluses).  If 
the  score  is  anything  less  than  three  pluses  for  the  optimal  path,  then 
100  percent  cannot  be  achieved. 

The  courseware  author  has  the  options  of  performing  the  hex  weighting 
operation  with  or  without  using  the  FSTT  as  an  aid.  It  Is  recommended 
that  the  FSTT  be  used  if  at  all  possible.  The  reason  is  that  the  FSTT 
provides  for  immediate  access  to  the  frame  requested,  which  has  been 
selected  from  the  image  already  being  displayed. 
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Weighting 

Code 


Meaning 

Item  not  used  for  scoring. 


Computational 

Value 


0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

A 

B 

C 

D 

E 

F 


Irrelevant  ( — ) 
Irrelevant  (— ) 
Irrelevant  (-) 

Neutral  (relevant) 

Relevant  (+) 
Relevant  (++) 
Relevant  (+++) 

Unsafe 

Incorrect  ( — ) 
Incorrect  (--) 
Incorrect  (-) 

Neutral  (correct) 

Correct  (+) 

Correct  (++) 
Correct  (+++) 


0 

1 

2 

3 

4 

5 

6 

0 

0 

1 

2 

3 

4 

5 

6 


Coding  the  Mag  Card 

The  author  will  load  in  the  fiche  for  the  problem  and  press  00  to 
start.  At  this  point,  the  overall  diagram  of  the  equipment  is  displayed 
and  the  options  for  selecting  troubleshooting  strategies  and/or  making 
observations  are  presented.  The  author  should  then  systematically  request 
the  frames  that  represent  exercising  the  optimal  path  for  solving  the 
problem.  (This,  of  course,  has  been  done  prior  to  obtaining  the  photo¬ 
graphs  in  the  first  place,  the  data  for  which  are  in  the  field  data 
collection  workbooks.)  The  printer  should  be  turned  on  because  it  will 
provide  a  convenient  record  of  the  path  taken  through  the  problem.  Once 
the  author  has  determined  that  an  optimal  path  has  been  established,  then 
the  values  7  or  F,  depending  on  whether  the  choice  is  relevant  or  correct, 
should  be  assigned  to  each  of  the  trainer  codes  in  the  path.  This  is  best 
done  by  entering  the  data  Immediately  onto  the  Mag  Card  Coding  Guides,  an 
example  of  which  can  be  found  on  the  final  three  pages  of  this  Appendix. 

The  correct  repair  must  be  selected  to  solve  the  problem,  followed  by 
a  verification  that  a  repair  has  been  made.  Finally,  the  user  must  declare 
that  a  solution  has  occurred.  Courseware  authorship  includes  coding  each 
frame  for  the  condition  that  exists  when  there  is  a  malfunction  and  for 
the  condition  that  exists  when  the  malfunction  has  been  cleared.  Thus, 
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for  each  frame  there  are  two  locations  on  the  magnetic  coding  guide  for 
inserting  hex  weighting  codes.  The  space  to  the  right  Is  the  one  to  use 
when  the  repair  has  not  yet  been  made.  The  one  on  the  left  is  the  place 
for  the  hex  weight  under  the  condition  that  a  repair  has  been  made.  Just 
because  there  are  two  locations  in  which  to  place  hex  weights,  it  does  not 
mean  that  the  hex  weights  need  be  different.  That  is,  it  may  be  efficient 
to  make  a  measurement  at  a  certain  location  when  trying  to  correct  the 
fault  (before  repair)  and  also  when  trying  to  verify  that  the  fault  has 
been  corrected  (after  repair).  In  that  case,  the  coding  for  the  second 
time  through  (for  verification)  should  be  negative  even  though  it  was 
positive  the  first  time  through. 

It  is  not  necessary  to  enter  Os  for  those  frames  not  used  in  the 
problem.  If  the  code  for  that  frame  is  entered  accidentally  the  FSTT  will 
display  a  frame  which  does  not  have  a  picture  on  it,  but  there  will  be  no 
effect  upon  the  score. 


Verification  Frames 

Recause  it  is  important  to  ensure  that  work  has  resulted  in  an  effec¬ 
tive  repair,  the  FSTT  provides  for  four  verification  frames.  The  author 
chooses  how  many  of  these  will  be  used  for  a  particular  problem,  and  then 
inserts  the  codes  associated  with  these  frames  In  Line  1,  Rows  16  through 
23  of  the  mag  card.  The  purpose  of  the  verification  frames  Is  to  require 
the  user  to  access  those  frames  prior  to  declaring  the  problem  to  be 
solved  (CC).  The  FSTT  will  present  a  message  "solved,  so  many  minutes, 
such  and  such  a  score"  if  the  user  accesses  those  verification  frames.  If 
the  user  falls  to  access  one  or  more  of  the  verification  frames,  a  correct 
solution  to  the  problem  will  not  be  given.  That  Is,  even  though  a  correct 
repair  is  made,  no  credit  for  a  solution  Is  given.  If  the  author  elects 
not  to  use  verification,  then  no  frames  need  be  accessed  during  the  veri¬ 
fication  chec*. 

Once  the  coding  for  the  optimal  path  has  been  determined  and  entered 
on  the  magnetic  coding  guide,  the  remainder  of  the  frames  available  in  the 
problem  should  be  coded. 

The  next  section  describes  the  forms  required  and  the  procedures  to 
use  in  actually  performing  the  coding  operation. 


Mag  Card  Authoring 
1 .  Data  source 

Work  from  Trainer  Code  Assignment  Sheet  (original  problem  workbook 
or  problem  architecture  document). 

a.  Each  frame  used  in  a  fiche  must  be  coded. 
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b.  Enter  data  onto  Mag  Card  Coding  Guide  in  pencil. 

Code  In  this  sequence 

a.  Header 

b.  Malfunction  present 

(1)  Optimal  path 

(a)  Code  optimal  path  to  solution  as  Fs  or  7s  to  ensure 
a  possible  100  score. 

(b)  Identify  verification  codes  (up  to  four  OK;  code  as 
7  or  F). 

(2)  Code  suboptimal  path  with  mix  of  weights  (author  judgment). 

(3)  Wrong  path 

(a)  Code  false  trails. 

(b)  Code  incorrect  or  hazardous  actions. 

c.  Malfunction  absent  (repair  made) 

(1)  Optimal  path 

(a)  Select  subset  of  frames  as  minimum  for  verification. 

(b)  Code  these  verification  frames  as  7  or  F  on  second 
run  through.  Students  must  select  these  verification 
frames;  hence,  full  weight  for  these  is  needed. 

(2)  Suboptimal  path 

(a)  Select  frames  that  are  not  absolutely  essential  for 
verification  but  are  not  irrelevant/incorrect. 

(b)  Code  as  less  than  optimal. 

(3)  Wrong  paths 

(a)  Select  frames  that  are  irrelevant  or  incorrect  for 
verification. 

(b)  Code  a  lower  Mag  Card  Code  for  all  wrong  path  choices 
on  second  run  through  since  these  codes  should  not  be 
selected  at  all. 


When  ready  to  code,  take  Mag  Card  Coding  Guide  (MCCG)  to  FSTT 

a.  Prepare  to  code  mag  card. 

(1)  Get  blank  mag  card.  (If  no  blank  card  Is  available, 

"zero"  a  card  by  recording  over  the  one  that  Is  data- 
filled.  "Zero"  a  card  by  entering  the  AUTHOR  mode  and 
entering  "0"  for  every  column  on  every  row.  When  all 
rows  contain  "zeros,"  press  "record."  This  will  record 
“0"  onto  the  mag  card  in  the  reader.  Do  this  for  all 
four  stripes.)  This  card  will  be  the  zero  card.  It 
should  be  used  in  PRACTICE  mode  to  turn-on  the  FSTT, 
prior  to  entry  to  AUTHOR  mode.  Start  with  a  blank  card 
so  that  no  extraneous  data  can  misrepresent  the  problem 
being  simulated. 

b.  With  the  mag  card  in  place,  turn  to  AUTHOR  mode.  The  line 
display  will  show  the  cursor  at  Row  0,  Column  0.  The  LD 
should  read  000000000000000.  Move  the  cursor  to  the  position 
of  the  first  data  to  enter,  probably  Column  3.  It  is  best 

to  use  the  FWD  key  to  do  this,  although  the  0  key  may  be  used. 
There  is  no  need  to  enter  zero.  When  "Record"  Is  pressed, 
the  mag  card  is  written  with  the  data  that  appear  on  the  LD. 

If  zeros  are  present,  they  are  recorded  as  zeros.  The  TAB 
key  moves  the  cursor  to  the  right  five  spaces  at  a  time. 

The  SCROLL  key  moves  one  line  at  a  time,  with  the  cursor  In 
the  same  column.  The  LINE  number  key  moves  the  cursor  to  a 
specific  line  number,  but  scrolling  does  too  and  may  be 
easier.  Mistakes  are  corrected  by  backing  over  the  error 
and  keying  correctly. 

c.  Enter  all  data,  row  by  row.  (It  is  a  one-person  job,  but  can 
be  speeded  up  if  one  person  reads  the  data  and  another  enters 
it.  NOTE:  All  keys  require  a  firm  force  to  activate  them. 

At  times,  it  seems  as  though  an  entry  was  made  but  only  the 
second  keystroke  appears  on  the  LD.  When  in  the  AUTHOR  mode, 
simply  backspace  over  the  incorrect  code  and  enter  the  correct 
one.  When  in  the  PRACTICE  mode,  simply  depress  the  cancel 
key.  The  result  is  an  X  immediately  beside  the  digit 
displayed.  Both  digits  may  then  be  reentered.) 

d.  When  finished,  press  "Print"  to  get  a  hard  copy  of  the 
keystrokes  entered. 

(1)  Verify  keystrokes  by  comparing  the  MCCG  to  the  printout. 

(2)  Make  corrections  as  needed. 

e.  Press  "Record."  This  records  the  data  on  a  card.  Now  may 
be  a  good  time  to  copy  all  stripes. 
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f.  Verify  data  are  recorded. 


(1)  Press  “Load." 


The  mag  card  can  be  re-recorded  as  follows: 

1.  Insert  mag  card  in  AUTHOR  mode. 

a.  Enter  "BKSPACD"  to  get  into  normal  edit. 

b.  Line  display  will  then  show  the  contents  of  Line  “00." 

2.  On  line  "00"  enter  data  of  this  revision,  if  desired.  (This  space  was 
intended  for  noting  changed  fiche  frames,  but  could  also  be  used  to 
record  other  revisions.)  Similarly,  space  for  Lesson  SN  is  also 
available  for  this  record.  Get  Line  #01  (use  SCROLL  key). 

3.  Use  FWD  key  to  move  cursor  to  Column  1. 

4.  Enter  code  (cursor  moves  to  next  column). 

5.  Enter  remainder  of  reinforcement  code. 

6.  Replace  mag  card  with  spare  (to  retain  original.) 

7.  a.  Press  "RECORD"  key  and  watch  in-line  display, 
b.  Press  "LOAD"  key  and  watch  in-line  display. 

8.  When  display  shows  Line  00,  the  newly  recorded  stripe  has  been 
reloaded.  Recording  Is  finished. 

9.  To  ensure  backup  of  this  code,  remove  and  reinsert  card  with  another 
stripe  Into  reader. 

10.  Press  RECORD.  (Do  this  for  the  remainder  of  the  stripes.) 


PROCEDURE  -  FORMS  REQUIRED 


Mag  Card  Coding  Guide 

The  FSTT  has  been  designed  to  handle  three  different  kinds  of  problems 
The  first  and  most  common  type  is  the  "single  malfunction"  type  of  problem 
The  second  Is  called  the  "compound  malfunction"  and  the  third  Is  the 
"no-malfunction"  problem. 

Magnetic  Card  Coding  Guides  were  developed  for  each  problem  type  and 
fiche  layout.  For  the  single  malfunction  (for  many  trainer  codes),  two 


adjacent  locations  on  the  fiche  are  accessed  by  a  given  trainer  code.  One 
location  is  for  the  system  with  the  malfunction  present;  the  condition  is 
for  the  system  free  of  trouble.  Depending  on  the  nature  of  the  malfunction, 
the  image  to  be  located  in  the  related  locations  may  be  the  same  or  may  be 
different.  In  the  case  of  a  no-malfunction  problem,  there  is  but  one 
image  to  show  under  all  circumstances  for  a  given  trainer  code. 

In  the  case  of  the  compound  malfunction,  four  physically  adjacent 
locations  are  available  for  display  of  images  associated  with  a  specific 
trainer  code.  These  locations  are  filled  as  follows: 

1.  A  location  at  which  the  image  to  be  displayed  is  that  of  the 
measurement  test  point  when  both  troubles  A  and  B  are  present 
simultaneously  in  the  system. 

2.  When  one  of  the  troubles  has  been  repaired. 

3.  When  the  other  one  of  the  troubles  has  been  repaired. 

4.  When  no  troubles  exist  in  the  system  at  all. 


Data  Entry 

All  data  relating  to  the  identification  of  a  given  problem,  and  the 
weighting  codes  associated  with  the  user's  choice  of  fiche  frame  to  view 
must  be  entered  onto  the  magnetic  stripe  associated  with  the  problem. 
This  is  done  during  the  authoring  phase. 


Data  Source 


The  workbook,  prepared  as  both  a  field  data  collection  and  authoring 
workbook,  contains  the  pages  organized  to  record  the  needed  data. 


Fiche  Assignment  Procedures 

When  the  author  has  specified  a  set  of  measurements  and/or  observations, 
to  make  for  a  given  problem,  the  related  photographs  must  be  assigned  to 
specific  frame  locations  in  each  fiche.  While  there  is  considerable 
freedom  to  assign  such  locations,  certain  constraints  do  exist.  For 
example,  certain  locations  (the  second  and  third  rows  of  Fiche  A  and  B, 
and  the  fourth  and  fifth  rows  of  B)  are  tied  together  by  the  design  of  the 
system.  If  the  user  selects  frames  within  this  "system  drawings"  area, 
both  viewers  slew  such  that  related  Images  are  displayed.  Asking  for 
assistance  (AA),  brings  up  the  related  frame  (from  the  fourth  or  fifth  row 
of  B)  on  the  right-hand  viewer.  It  Is  Important  that  the  courseware  author 
use  these  dedicated  fiche  frame  locations  correctly.  Nothing  prevents 
misuse,  of  course.  The  problem  is  that  if  the  frames  are  misused,  they 
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might  not  be  available  when  needed.  Likewise,  test  and  measurement  (7W) 
frames  (located  predominantly  in  Fiche  C)  have  the  characteristic  that  when 
a  repair  action  changes  the  condition  of  the  problem  (makes  bad  signal 
good  because  a  repair  has  been  made),  the  FSTT  then  causes  a  different 
frame  to  be  displayed  in  place  of  the  frame  containing  a  photograph  of  a 
TM  display  Illustrating  the  malfunction  condition.  Clearly,  selecting  one 
of  the  TAM  frames  to  contain  system  level  data  would  not  be  useful. 


Entering  Codes 

Once  the  weights  have  been  allocated  to  the  frames,  they  must  then  be 
entered  Into  the  magnetic  stripe. 

The  first  column  Identifies  the  line  number.  (When  the  mag  card  is 
Inserted  into  the  card  reader  and  the  author  mode  has  been  selected,  the 
line  display  should  light  up,  showing  the  cursor  underline  in  the  first, 
leftmost  position.  This  Is  position  at  nibble  "0".  At  the  extreme  left 
the  display  will  show  "LOO  COO"  meaning  L=line  #  "00"  and  OColumn  #) . 

The  heading  of  the  second  column  Is  "nibble."  This  column  specifies  which 
nibbles  are  associated  with  the  data  element  shown  In  the  third  column, 
"content."  In  this  example,  the  data  entries  for  line  00,  nibbles  00 
through  15  are  shown.  As  Indicated,  the  simulator  ID  is  given  in  nibbles 
00-03.  (The  digits  that  appear  in  the  line  display  each  occupy  one  display 
position.  The  display  Is  capable  of  displaying  32  alphanumeric  digits. 
Although  the  display  has  been  designed  to  show  any  ASCII  charcter,  each 
character  position  has  only  a  half-byte  associated  with  it;  l.e.,  four 
bits  when  authoring.  Thus,  the  codes  that  may  be  Interpreted  by  the 
microprocessor  can  be  any  of  the  16  hexldecimal  digits,  0-F.  Although  the 
display  will  provide  for  the  showing  of  messages  in  clear  text,  such  as 
"UNSAFE,"  when  the  display  Is  used  during  the  authoring  phase,  it  will 
merely  display  the  values  of  the  hex  codes  Inserted,  i.e.,  0-F.) 

The  example  Identifies  the  simulator  as  Number  1.  (This  number  could 
be  some  part  of  the  specific  simulator  Identification  number  at 
Plattsburgh.)  However,  since  the  problems  are  developed  in  support  of  a 
particular  configuration  of  equipment.  It  is  important  to  specify  the 
simulator  to  which  the  problem  refers. 


Line  00 


Nibbles  04-09  Nibbles  04  through  07  are  to  be  used  in  identifying  the 
lesson.  In  the  example,  the  lesson  Serial  Number  is  "4," 
shown  in  position  7.  Nibbles  08  and  09  identify  the 
suffix  code  to  be  used.  This  code  specifies  which  fiche 
will  be  pulled  from  the  cartridge  for  the  problem. 


Nibbles  10- 

Line  01 
Nibble  00 

Nibble  01 

Nibble  02 

Nibble  03 
Nibbles  04, 

Nibbles  06, 


Nibbles  08, 


15  These  nibbles  provide  for  entering  the  month,  day,  and 
year  of  the  revision  (or  edition)  of  the  problem.  (It 
is  assumed  some  coding  changes  will  occur  because  of 
improvements  in  the  design  of  the  problem--or  hardware 
charges  in  the  equipment  being  simulated--fl ight 
simul  ator .) 


Specifies  number  of  malfunctions  in  the  problem  (0,  1,  or 
2).  In  the  example,  the  number  "1"  is  inserted,  identi¬ 
fying  this  problem  as  being  a  "single  malfunction  problem." 

The  first  digit  in  the  reinforcement  condition  set.  It 
is  the  "message  code." 

The  second  digit  in  the  reinforcement  condtion  set.  It 
is  the  "schedule  code." 

The  "score  code"  specifies  conditions  for  displaying  the 
score. 

05  Specifies  the  fault  identification  code  for  malfunction 
A.  In  the  example,  F9  is  the  ID.  (When  F9  is  keyed  by 
the  user,  the  repair  Menu  #1  on  the  leftmost  viewer  is 
displayed,  and  the  FSTT  is  conditioned  to  receive  the 
keystrokes  associated  with  nibbles  08  and  09,  provided 
that  the  frame  named  in  nibbles  06  and  07  has  been 
accessed  first. 

07  Fault  frame  -  malfunction  A.  The  trainer  code  entered 
here  is  the  one  which  will  cause  display  of  the  frame 
containing  the  malfunctioning  part.  It  is  required  that 
the  trainee  key  this  trainer  code  prior  to  keying  in  the 
fault  code.  Although  the  fault  code  can  be  keyed  at  any 
time,  the  correctness  of  the  problem's  solution — i.e., 
credit  for  using  sound  troubleshooting  methods--will  be 
given  only  on  the  condition  that  the  fault  frame  be 
screened  first.  This  prevents  a  trainee  from  solving 
problems  by  simply  entering  all  the  fault  codes  and  the 
available  repair  options. 

09  Correct  repair  code  -  malfunction  A.  The  code  entered 
here  is  the  switch  that  turns  otherwise  bad  signals  to 
good  ones.  If  an  author  definable  code  (ADC)  Is  used, 
then  no  physical  change  appears  to  follow  when  this  code 
is  entered.  However,  if  the  code  is  a  trainer  code,  a 
viewer  could  easily  slew  to  an  image  not  seen  previously. 
Therefore,  the  software  is  designed  to  make  the  switch 
prior  to  slewing  to  the  new  view. 


Ill 


Nibbles  10-15  Unused. 


Nibbles  16-23  Verification  Frames.  The  code  entered  here  is  the  trainer 
code  of  frames  (up  to  four)  that  must  be  seen  following 
"00"  after  a  repair  has  been  made.  A  100  percent  solution 
will  not  be  awarded  unless  the  trainee  accesses  these 
frames  (if  used) . 


Line  02 

Nibbles  00-15  Lesson  Stop  Codes.  These  16  positions  are  locked  to  the 
eight  messages  that  can  abort  a  lesson.  There  are  16 
nibbles  available  because  each  pair  of  nibbles,  while 
related  to  a  specific  message,  may  contain  either  "00",  in 
which  case  no  message  would  be  provided,  or  they  may  con¬ 
tain  an  "author  definable  hex  code."  In  the  latter  case, 
the  input  of  this  code  from  the  keypad  would  result  in 
display  of  the  selected  message.  The  ADC  would  be 
accessed  by  the  trainee  from  some  photograph  that  would 
provide  it  as  a  part  of  the  display,  or  it  could  be  the 
result  of  a  menu  page  which  provided  options  for  repair 
actions.  In  the  example,  lesson  stop  codes  for  nibbles  08 
and  09  were  inserted.  This  is  for  "possible  Injury"  and 
"A4"  was  selected.  When  A4  is  keyed,  the  problem  aborts. 
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APPENDIX  C  -  SAMPLE  PROBLEM 


Students  use  this  Sample  Problem  as  a  guide  for  how  to  use  the  FSTT  in 
solving  other  FSTT  troubleshooting  problems.  The  guidance  is  designed  to 
tell  the  user  what  codes  to  enter  and  why.  The  user  is  guided  through  an 
efficient  path  to  solution.  Of  course,  the  user  may  choose  to  stray  from 
the  guidance,  may  use  it  more  than  once,  or  may  ignore  the  guidance.  The 
words  of  guidance  below  the  frames  in  the  printed  version  do  not  appear  on 
the  FSTT  screens.  They  appear  only  on  this  copy.  The  frames  shown  on  the 
printed  Sample  Problem  are  frames  that  do  come  up  on  the  FSTT  as  the  user 
works  his  way  through  the  sample.  No  other  FSTT  problems  appear  in  a 
guided  version. 
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frame  which  lists  other  problems  available  in  the  FSTT  library.  You  can  compare  your  score  against 
a  scale  available  from  your  instructor  (supervisor).  This  varies  from  problem  to  problem.  You  may 
wish  to  redo  the  problem  to  get  a  higher  score.  It  wouldn't  hurt.  The  Printer  will  show  you  the 
path  you  took  to  solve  the  problem. 


APPENDIX  D  -  DESCRIPTION  OF  FSTT  FAILURES  AND  CORRECTIVE 
ACTIONS  TAKEN  DURING  FORMATIVE  EVALUATION 
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APPENDIX  D  -  DESCRIPTION  OF  FSTT  FAILURES  AND  CORRECTIVE 
ACTIONS  TAKEN  DURING  FORMATIVE  EVALUATION 

HARDWARE  PROBLEMS  CORRECTED  DURING  FORMATIVE  EVALUATION 
Microfiche  Viewers 


The  microfiche  viewers  had  three  kinds  of  problems.  First,  one  of  the 
two  printed  circuit  boards  in  each  of  the  viewers  suffered  a  burned  out 
power  transistor.  The  problem  was  routinely  solved  and  a  repair  made. 

This  board  is  in  the  circuitry  which  extracts  fiche  from  the  cartridges. 
Unless  the  fiche  can  be  pulled  into  the  viewer,  it  cannot  be  seen. 

Another  problem  was  that  the  fiche  and  their  jackets  were  being  mangled  by 
the  mechanical  innards  of  the  viewers.  While  this  did  not  happen  very 
often,  it  justified  obtaining  both  multiple  copies  of  fiche  (10  per 
problem)  and  extra  carriers. 

All  fiche  in  the  cartridges  are  contained  within  transparent  mylar 
carriers.  When  a  fiche  is  selected,  a  nylon  "prod"  moves  to  the  correct 
position  for  that  fiche  and  strikes  a  tab  that  is  part  of  the  carrier. 

This  causes  the  carrier  to  move  about  1/8  inch  into  the  viewer  to  contact 
a  set  of  rollers  which  withdraws  the  carrier  into  the  viewer.  These 
rollers  are  a  source  of  dirt  distribution  in  the  system.  If  any  carrier 
should  have  an  oily  smudge  on  it,  some  part  of  the  dirt  will  be  trans¬ 
ferred  to  the  rollers.  Thereafter,  the  roller  will  transfer  a  part  of  the 
dirt  to  other  carriers  and  so  on.  This  means  that  carriers  should  be  kept 
free  of  dirt,  and  as  dust  free  as  possible  before  use,  and  should  be 
cleaned  periodically.  Replacement  of  the  rollers  and  carriers  would  be  a 

Repeated  strikes  on  the  tabs  of  the  carriers  tends  to  wear  them  out. 

If  the  carriers  are  not  replaced  soon  enough,  a  time  will  come  when  the 
carrier  will  not  be  drawn  into  the  viewer  because  the  tab  is  bent  and  does 
not  transmit  the  force  needed  to  move  the  carriers.  Periodic  (monthly) 
inspection  of  these  tabs  will  effectively  identify  those  which  are  wearing 
out  and  which  can  then  be  replaced. 

The  carriers  (Consolidated  Micrographics,  successor  to  Bruning)  were 
purchased  in  sets  of  30  carriers.  The  cartridge  has  this  capacity; 
however,  in  the  FSTT  application,  the  cartridges  are  loaded  with  only  15 
carriers.  There  are  15  spares  available.  By  removing  the  carriers  for 
even  numbers,  and  turning  them  over,  an  additional  set  of  odd  numbered 
carriers  is  obtained.  Hence,  because  of  how  the  fiche  carriers  are  used 
(15  per  cartridge),  a  set  of  30  spare  carriers  actually  provides  two  sets 
of  spares. 

As  mentioned  previously,  carriers  can  be  mangled  by  the  mechanical 
handling  system.  A  possible  cause  of  this  Is  poor  positioning  of  the 
fiche  internal  to  the  carrier.  If  pert  of  the  fiche  extends  beyond  the 
edge  of  the  carrier,  it  can  be  caught  askew  to  the  rollers,  causing 
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entrance  to  the  platen  at  an  angle.  This  will  cause  either  the  flche  or 
the  carrier  to  be  mangled.  If  that  occurs,  the  FSTT  must  be  shut  down  and 
the  materials  removed  from  the  viewer  Interior.  A  replacement  flche  and 
carrier  then  need  to  be  Installed.  Slice  five  copies  of  each  flche  were 
produced  and  spare  carriers  were  provided,  no  serious  problem  resulted 
from  this  cause. 

Users  of  the  FSTT  were  not  really  concerned  about  dirt  in  the  visual 
path,  but  it  does  not  make  a  good  Impression  on  casual  observers  of  the 
device.  There  are  several  places  where  dirt  can  enter  and  possibly  cause 
a  problem.  Dust  can  settle  Into  the  optical  path  mirrors  and  lenses. 

Dust  Is  easy  to  remove  from  these  surfaces,  using  a  dry  air  jet  or  lens 
tissues.  Dust  and  dirt  can  also  be  deposited  on  the  fiche  themselves. 

Even  though  each  flche  Is  In  Its  own  paper  jacket  prior  to  loading  In  a 
carrier,  some  dust  matter  can  land  on  them  and  particles  of  dust  sometimes 
show  up  In  the  projected  Image.  As  a  result.  It  was  found  that  use  of 
nylon  gloves  (lint-free  gloves)  was  helpful  in  handling  the  fiche  and 
carriers.  Powder  or  oils  must  not  be  used  to  Increase  sliding  of  one 
carrier  over  the  other.  There  Is  some  sticking  of  carriers  which  results 
In  two  or  more  carriers  being  withdrawn  at  once;  It  is  caused  by  a  buildup 
of  static  electricity.  While  this  buildup  Is  difficult  to  prevent.  It  is 
easy  to  overcome.  The  user  should  merely  press  on  the  tabs  when  the 
cartridge  Is  out  of  the  viewer  and  gently  move  the  carriers  against  them¬ 
selves  and  manually  touch  the  plastic  edges  of  the  carriers.  Using  a 
photographer's  brush  (with  radioactive  element)  will  discharge  the  carrier 
during  loading  to  minimize  electrostatic  attractions  of  dust  during  this 
phase  of  use. 


Alignment 

During  hardware  development,  tests  on  the  repeatability  of  viewer  pre¬ 
sentations  were  made.  It  Is  desirable  for  the  Image  displayed  to  fall  on 
the  same  physical  place  on  the  screen  for  each  frame.  These  tests  showed 
that  the  viewer  was  able  to  position  the  Image  within  ±  1/4  inch. 
Accordingly,  a  mask  was  designed  to  hide  this  amount  of  error  from  view. 
Implications  for  copy  preparation  are  that  codes  need  to  be  placed  well 
within  the  edges  of  a  frame,  lest  they  be  cut  off  partially  or  wholly  by 
the  mask.  Variability  In  Image  presentation,  however,  was  not  due 
entirely  to  electromechanical  viewer  flche  positions.  The  flche  themselves 
need  to  be  positioned  within  the  carriers  at  fixed  locations. 

Two  flche  producers  were  used  in  making  the  fiche.  Each  produced  fiche 
with  slight  variations,  both  from  flche  to  flche  and  from  manufacturer  to 
manufacturer.  Reducing  this  variability  was  done  through  cutting  the 
flche  material  to  standard  dimensions.  Ultimately,  it  was  evident  that 
the  manual  positioning  controls  on  the  viewers  themselves  were  needed  and 
In  fact,  were  used. 
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As  previously  stated,  the  purpose  of  the  calibration  card  was  to  load 
a  table  of  values  that  would  instruct  the  viewer  as  to  where  to  position 
its  fiche  transport  to  display  a  frame.  Although  calibrations  can  be 
quite  exact  for  a  given  "MASTER"  fiche,  there  are  enough  other  variables 
of  non-standard  type  whi ch  effectively  offset  the  fine-tuning  of  the 
calibration.  With  care,  standardized  fiche  can  be  produced  and  loaded. 
But  the  friction  factors  in  the  mechanical  system  will  always  impose  the 
need  for  minor  adjustments  in  positioning. 

Obviously,  a  feedback  servo-positioning  system  could  be  designed  to 
overcome  this  registration  problem.  It  would  not,  however,  be  off-the- 
shel f . 

The  preceding  discussion  notwithstanding,  users--both  the  instructors 
and  the  three- level  airmen— were  not  at  all  concerned  with  "dirt"  in  the 
visual  path,  or  the  need  to  manually  position  a  frame. 


Readability  of  Access  Codes 

The  original  access  code  design  consisted  of  black  or  white  inked  hex 
digits  on  transparent  plastic  peel -off,  self  adhesive  labels.  These  were 
used  in  the  first  few  problems  produced.  These  codes  were  found  to  have 
insufficient  contrast  and  were  discontinued  in  favor  of  opaque  hex  codes 
of  the  same  size. 

Another  change  was  made  to  use  codes  of  almost  twice  the  original  size 
to  enhance  the  readability.  Black  print  on  white  backgrounds  was  used 
when  the  image  was  darkly  hued.  This  requirement  was  imposed  by  the  color 
microfiche  maker  who  tended  to  wash  away  the  black  print  on  white 
backgrounds  if  used  on  dark  hues  as  a  result  of  the  longer  time  exposures 
required  to  copy  the  dark-hued  materials.  (Some  of  the  photographs  of 
simulator  cabinets  resulted  in  high  contrast  pictures  which  were  predomi¬ 
nantly  dark.  Long  exposures  were  needed  to  capture  the  detail  that  was 
needed.) 


Aids  to  Problem  Solution 

A  special  set  of  aids  was  developed  for  each  problem  consisting  of  one 
printed  page  per  problem.  These  are  included  in  Appendix  A.  They  serve 
to  provide  the  reader  with  a  description  of  each  problem.  No  revisions 
were  made  to  them  during  formative  evaluation  as  they  did  not  appear  to  be 
used  much  by  FSTT  users,  and  no  comments  were  made  about  them. 


Miscellaneous  Observations 

The  photographs  shown  as  FSTT  images  provide  a  self- revealing  record 
of  flight  simulator  status  for  the  abnormal  and  normal  conditions.  It  was 
not  necessary  to  state  the  conditions  of  switches  and  meters  and  scopes — 


the  states  are  revealed  by  their  own  positions  when  the  user  calls  up  the 
picture  with  the  switch  settings  shown  on  it.  Each  problem  used  an  ini¬ 
tial  frame  that  showed  a  Form  781  write-up.  It  was  expected  that  on  the 
job  a  repairman  would: 

1.  Discover  what  the  write-up  meant. 

2.  Verify  its  accuracy. 

3.  Search  for  the  cause. 

4.  Repair. 

5.  Verify  the  repair. 

Users  routinely  performed  all  these  functions,  except  the  last.  They 
did  not  recognize  that  they  had  to  verify  that  their  repair  had  cleared 
all  symptoms  of  malfunction.  This  verification  is  an  essential  step  on 
the  job.  It  may  be  that  users  did  not  initially  realize  that  the  FSTT 
required  this  function  just  as  the  job  did.  However,  the  scoring  display 
reads  "not  finished"  if  the  user  does  not  verify  after  making  the  replace¬ 
ment  of  the  bad  part  (even  though  the  bad  part  has  been  replaced).  Some 
users  complained  that  this  "robbed"  them  of  credit  for  a  solution. 

However,  users  of  the  FSTT  learned  that  verification  was  part  of  the  job 
function  and  that  the  FSTT  required  the  function  to  be  performed. 

On  problems  with  multiple  malfunctions,  the  symptoms  of  the  second 
malfunction  would  remain  after  the  first  malfunction  was  verified. 

With  the  FSTT,  the  users  are  permitted  to  move  about  the  system  and  to 
make  measurements  according  to  their  own  desires.  Some  users  tended  to  be 
reluctant  to  make  a  selection  at  first.  There  are  general  reasons  for 
this: 

1.  Fear  of  making  a  mistake  and  the  ensuing  embarrassment. 

2.  Awkwardness  in  a  new  environment. 

3.  Inability  to  select  a  measurement  which  they  desire  but 
which  Is  not  available  In  the  Image  being  viewed. 

As  for  fear  of  making  an  error.  It  was  found  that  most  users  get  over 
this  quickly  on  the  FSTT.  Curiously  It  was  observed  that  In  multiple 
trainee  situations,  the  presence  of  a  peer  encourages  a  certain  banter, 
which  in  turn,  tends  to  make  the  user  more  willing  to  take  a  chance. 

Awkwardness  In  the  new  environment  Is  a  common  phenomenon.  In  the 
case  of  the  FSTT,  the  user  sees  access  codes  with  square,  circular, 
diamond- shaped,  and  triangular  surrounds.  Now,  practically,  these 
distinctions  in  shape  do  not  make  any  difference  since  keying  a  unique 


hex  code  will  display  whatever  is  on  the  related  frame.  There  is  no  dif¬ 
ference  in  scoring  based  on  these  shapes.  The  shape  merely  aids  the  user 
in  choosing  the  type  of  action  to  be  taken.  This  awkwardness  was  readily 
overcome. 

There  is  a  limit  to  the  number  of  access  codes  that  can  be  placed  on 
one  image.  In  the  real  world  the  number  of  actions  that  can  be  taken  is 
much  larger.  However,  only  a  few  are  reasonable.  The  FSTT  has  a  reason¬ 
able  number  of  choices. 
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